Repository: dbt-labs/dbt-core

Issue:
invocation_id not the same through the whole dbt run.
### Describe the bug
Running: dbt run --m tag:x

48models are run, with a pre-hook and post hook which logs the start and end of each model, and a on-run-start and on-run-end hook which logs the start and end of the run.

After about 35models are run, the error message: Error sending message, disabling tracking is displayed, but the run keeps on going.

After the run dbt has inserted all model starts, ends and the start of the run with the same Invocation_id, but the on-run-end hook insert, which tells that the run was completed has a different invocation_id.

This only happens if I run a dbt which loads many models, not if I run dbt run --m speficic model.

This does happen when I run dbt locally, but not if I run the same dbt project from our server we use to schedule dbt runs.



### Steps To Reproduce
In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here.

-- Dont know

### Expected behavior
A clear and concise description of what you expected to happen.

The run to have the same invocation_id

### Screenshots and log output
20:54:46 | Running 4 on-run-start hooks
20:54:46 | 1 of 4 START hook: dbt_edw.on-run-start.0............................ [RUN]
20:54:46 | 1 of 4 OK hook: dbt_edw.on-run-start.0............................... [SUCCESS 1 in 0.26s]
20:54:46 | 2 of 4 START hook: dbt_edw.on-run-start.1............................ [RUN]
20:54:46 | 2 of 4 OK hook: dbt_edw.on-run-start.1............................... [SUCCESS 1 in 0.29s]
20:54:46 | 3 of 4 START hook: dbt_edw.on-run-start.2............................ [RUN]
20:54:47 | 3 of 4 OK hook: dbt_edw.on-run-start.2............................... [SUCCESS 1 in 0.30s]
20:54:47 | 4 of 4 START hook: dbt_edw.on-run-start.3............................ [RUN]
20:54:48 | 4 of 4 OK hook: dbt_edw.on-run-start.3............................... [SUCCESS 1 in 1.23s]

20:55:19 | 30 of 48 OK created table model ...... [SUCCESS 1 in 16.19s]
20:55:21 | 2 of 48 OK created table model ...... [SUCCESS 1 in 33.55s]
20:55:22 | 31 of 48 OK created table model ...... [SUCCESS 1 in 17.75s]
20:55:22 | 11 of 48 OK created table model ...... [SUCCESS 1 in 34.22s]
20:55:22 | 8 of 48 OK created table model ...... [SUCCESS 1 in 34.44s]
Error sending message, disabling tracking
20:55:23 | 3 of 48 OK created table model ...... [SUCCESS 1 in 35.18s]
20:55:23 | 34 of 48 OK created table model ...... [SUCCESS 1 in 16.67s]
20:55:24 | 44 of 48 OK created table model ...... [SUCCESS 1 in 8.51s]
20:55:25 | 4 of 48 OK created table model ...... [SUCCESS 1 in 36.99s]

21:00:31 | Running 2 on-run-end hooks
21:00:31 | 1 of 2 START hook: dbt_edw.on-run-end.0.............................. [RUN]
21:00:34 | 1 of 2 OK hook: dbt_edw.on-run-end.0................................. [SUCCESS 1 in 2.89s]
21:00:34 | 2 of 2 START hook: dbt_edw.on-run-end.1.............................. [RUN]
21:00:37 | 2 of 2 OK hook: dbt_edw.on-run-end.1................................. [SUCCESS 1 in 3.11s]

### System information
**Which database are you using dbt with?**
- [x] snowflake



**The output of `dbt --version`:**
``` 
0.16.0
```

**The operating system you're using:**
Windows 10

**The output of `python --version`:**
3.8.2
### Additional context
Add any other context about the problem here.



Discussion:

Resolving patch:
diff --git a/core/dbt/config/runtime.py b/core/dbt/config/runtime.py
--- a/core/dbt/config/runtime.py
+++ b/core/dbt/config/runtime.py
@@ -522,7 +522,7 @@ def _get_rendered_profile(
             # return the poisoned form
             profile = UnsetProfile()
             # disable anonymous usage statistics
-            tracking.do_not_track()
+            tracking.disable_tracking()
         return profile
 
     @classmethod
diff --git a/core/dbt/tracking.py b/core/dbt/tracking.py
--- a/core/dbt/tracking.py
+++ b/core/dbt/tracking.py
@@ -44,7 +44,7 @@ def handle_failure(num_ok, unsent):
         # num_ok will always be 0, unsent will always be 1 entry long, because
         # the buffer is length 1, so not much to talk about
         logger.warning('Error sending message, disabling tracking')
-        do_not_track()
+        disable_tracking()
 
     def _log_request(self, request, payload):
         sp_logger.info(f"Sending {request} request to {self.endpoint}...")
@@ -112,6 +112,12 @@ def initialize(self):
         subject.set_user_id(self.id)
         tracker.set_subject(subject)
 
+    def disable_tracking(self):
+        self.do_not_track = True
+        self.id = None
+        self.cookie_dir = None
+        tracker.set_subject(None)
+
     def set_cookie(self):
         # If the user points dbt to a profile directory which exists AND
         # contains a profiles.yml file, then we can set a cookie. If the
@@ -364,6 +370,14 @@ def flush():
     tracker.flush()
 
 
+def disable_tracking():
+    global active_user
+    if active_user is not None:
+        active_user.disable_tracking()
+    else:
+        active_user = User(None)
+
+
 def do_not_track():
     global active_user
     active_user = User(None)


Test patch:
diff --git a/test/unit/test_tracking.py b/test/unit/test_tracking.py
new file mode 100644
--- /dev/null
+++ b/test/unit/test_tracking.py
@@ -0,0 +1,72 @@
+import dbt.tracking
+import datetime
+import shutil
+import tempfile
+import unittest
+
+
+class TestTracking(unittest.TestCase):
+    def setUp(self):
+        dbt.tracking.active_user = None
+        self.tempdir = tempfile.mkdtemp()
+
+    def tearDown(self):
+        dbt.tracking.active_user = None
+        shutil.rmtree(self.tempdir)
+
+    def test_tracking_initial(self):
+        assert dbt.tracking.active_user is None
+        dbt.tracking.initialize_tracking(self.tempdir)
+        assert isinstance(dbt.tracking.active_user, dbt.tracking.User)
+
+        invocation_id = dbt.tracking.active_user.invocation_id
+        run_started_at = dbt.tracking.active_user.run_started_at
+
+        assert dbt.tracking.active_user.do_not_track is False
+        assert isinstance(dbt.tracking.active_user.id, str)
+        assert isinstance(invocation_id, str)
+        assert isinstance(run_started_at, datetime.datetime)
+
+        dbt.tracking.disable_tracking()
+        assert isinstance(dbt.tracking.active_user, dbt.tracking.User)
+
+        assert dbt.tracking.active_user.do_not_track is True
+        assert dbt.tracking.active_user.id is None
+        assert dbt.tracking.active_user.invocation_id == invocation_id
+        assert dbt.tracking.active_user.run_started_at == run_started_at
+
+        # this should generate a whole new user object -> new invocation_id/run_started_at
+        dbt.tracking.do_not_track()
+        assert isinstance(dbt.tracking.active_user, dbt.tracking.User)
+
+        assert dbt.tracking.active_user.do_not_track is True
+        assert dbt.tracking.active_user.id is None
+        assert isinstance(dbt.tracking.active_user.invocation_id, str)
+        assert isinstance(dbt.tracking.active_user.run_started_at, datetime.datetime)
+        assert dbt.tracking.active_user.invocation_id != invocation_id
+        # if you use `!=`, you might hit a race condition (especially on windows)
+        assert dbt.tracking.active_user.run_started_at is not run_started_at
+
+    def test_tracking_never_ok(self):
+        assert dbt.tracking.active_user is None
+
+        # this should generate a whole new user object -> new invocation_id/run_started_at
+        dbt.tracking.do_not_track()
+        assert isinstance(dbt.tracking.active_user, dbt.tracking.User)
+
+        assert dbt.tracking.active_user.do_not_track is True
+        assert dbt.tracking.active_user.id is None
+        assert isinstance(dbt.tracking.active_user.invocation_id, str)
+        assert isinstance(dbt.tracking.active_user.run_started_at, datetime.datetime)
+
+    def test_disable_never_enabled(self):
+        assert dbt.tracking.active_user is None
+
+        # this should generate a whole new user object -> new invocation_id/run_started_at
+        dbt.tracking.disable_tracking()
+        assert isinstance(dbt.tracking.active_user, dbt.tracking.User)
+
+        assert dbt.tracking.active_user.do_not_track is True
+        assert dbt.tracking.active_user.id is None
+        assert isinstance(dbt.tracking.active_user.invocation_id, str)
+        assert isinstance(dbt.tracking.active_user.run_started_at, datetime.datetime)