Repository: locustio/locust

Issue:
--no-reset-stats should be on by default 
Please make this on by default.  It is what most people will want in a realistic load test.  

Case in point. We are gradually hatching about 1000 locusts that all need to authenticate with a login server. We hatch them slowly to avoid overwhelming the login server. After each locust logs in,  it immediately starts performing our regular workflow. In our first test runs, we lost the stats that got collected during the hatch because we didn't know `--no-reset-stats` existed. Watching your stats reset to 0 is extremely baffling. We had to scour the interweb to learn about this setting, and then spent a few hours figuring out how the switch works in a master/slave scenario

Making this on by default begs the question of how someone would turn it back off if they really do want to reset their stats when the hatch completes. Some options:

1. Ditch `--no-reset-stats` and replace it with a switch that has the opposite polarity : `--reset-stats-on-hatch-complete`
2. Just let me hook `on_hatch_complete` and reset the stats on my own if that is the behavior I want. Assumes that I can do this in my locust class or task set.
 


Discussion:
I actually think this would be a good idea. I have fielded this question often.

Curious what other people think. @heyman you have thoughts on this?
The most confusion/frustrating thing is that no where in the current documentation does it mention that all your data will reset after all Locust have been hatched. 
Yeah I always run the tests with --no-reset-stats,I think it would be great if it set by default
Agreed, it ought to be the default setting. As previously mentioned, the documentation doesn't spell out what is happening, or indicate that the option to not reset exists. We ended up with many many invalid test results because of this settings before realizing what was going on.
Yeah, I'm all for it!
That is, alternative 1. Turn it off by default, and add a `--reset-stats` flag.
@heyman, I will try to knock out a PR for this 
resetting stats may not be intuitive behavior, but you get better results.

I'm curious why anyone would you want to mix metrics from the hatching phase into the main results?  Just beware that including this data from before the workload has stabilized, the result distribution can be skewed and any calculated stats may not be representative.

However, I'm fine changing the default as long as we keep a way to reset stats and document the switch to do so.

I personally feel like more data and a clear procedure would lead to "better results". While using locust, I was unaware that the beginning of my data was being truncated and this lead me to question my data and ultimately not trust it. 
 
> Just beware that including this data from before the workload has stabilized, the result distribution can be skewed and any calculated stats may not be representative.

This could also be the sole purpose of a test (to find the point of failure). So if I was stress testing over night with a slow ramp of users, by the time I came back to check my results later they could potentially be truncated. 

Im not opposed to the current way of truncating ramp data, but at least clearly tell me that you are doing that. Finding this option in the locust issues as opposed to the Documentation or a UI checkbox was frustrating. 

@cgoldberg in my experience people like comparing "requests made" vs "requests serviced" (from the server) and it's confusing if those numbers differ, which they do because the requests are reset.

Resolving patch:
diff --git a/locust/main.py b/locust/main.py
--- a/locust/main.py
+++ b/locust/main.py
@@ -216,9 +216,15 @@ def parse_options():
     parser.add_option(
         '--no-reset-stats',
         action='store_true',
-        dest='no_reset_stats',
+        help="[DEPRECATED] Do not reset statistics once hatching has been completed. This is now the default behavior. See --reset-stats to disable",
+    )
+
+    parser.add_option(
+        '--reset-stats',
+        action='store_true',
+        dest='reset_stats',
         default=False,
-        help="Do not reset statistics once hatching has been completed",
+        help="Reset statistics once hatching has been completed. Should be set on both master and slaves when running in distributed mode",
     )
     
     # List locust commands found in loaded locust files/source files
diff --git a/locust/runners.py b/locust/runners.py
--- a/locust/runners.py
+++ b/locust/runners.py
@@ -44,7 +44,7 @@ def __init__(self, locust_classes, options):
         # register listener that resets stats when hatching is complete
         def on_hatch_complete(user_count):
             self.state = STATE_RUNNING
-            if not self.options.no_reset_stats:
+            if self.options.reset_stats:
                 logger.info("Resetting stats\n")
                 self.stats.reset_all()
         events.hatch_complete += on_hatch_complete


Test patch:
diff --git a/locust/test/test_parser.py b/locust/test/test_parser.py
new file mode 100644
--- /dev/null
+++ b/locust/test/test_parser.py
@@ -0,0 +1,26 @@
+import unittest
+
+from locust.main import parse_options
+
+
+class TestParser(unittest.TestCase):
+    def setUp(self):
+        self.parser, _, _ = parse_options()
+
+    def test_default(self):
+        opts, _ = self.parser.parse_args([])
+        self.assertEqual(opts.reset_stats, False)
+
+    def test_reset_stats(self):
+        args = [
+            "--reset-stats"
+        ]
+        opts, _ = self.parser.parse_args(args)
+        self.assertEqual(opts.reset_stats, True)
+
+    def test_should_accept_legacy_no_reset_stats(self):
+        args = [
+            "--no-reset-stats"
+        ]
+        opts, _ = self.parser.parse_args(args)
+        self.assertEqual(opts.reset_stats, False)