Repository: dbt-labs/dbt-core

Issue:
[CT-1367] New tracking event: Adapter info
We currently try to get the adapter information for tracking purposes long before we actually load the adapters-- this causes a bunch of issues where we have to parse projects and such just to track the invocation.  Since there's no benefit to this in terms of our analytics, this approach has [been changed in this PR](https://github.com/dbt-labs/dbt-core/pull/5972). 

This ticket encompasses the work necessary to properly report adapter info at such a point that we actually load the adapters.  This includes the work to update the internal analytics project to reflect the change in raw event data.

Specifically we need `adapter_type` and `adapter_unique_id` as indicated [here](https://github.com/dbt-labs/dbt-core/blob/79f043a733becd41459711fc5261ab1088deaa58/core/dbt/tracking.py#L470). 
[CT-1368] New tracking event: Project ID
We currently try to get the project ID for tracking purposes long before we actually load the project-- this causes a bunch of issues where we have to parse projects and such just to track the invocation.  Since there's no benefit to this in terms of our analytics, this approach has [been changed in this PR](https://github.com/dbt-labs/dbt-core/pull/5972). 

This ticket encompasses the work necessary to properly report the project ID at such a point that we actually load the project. This includes the work to update the internal analytics project to reflect the change in raw event data.

Specifically we need `project_id` as indicated [here](https://github.com/dbt-labs/dbt-core/blob/79f043a733becd41459711fc5261ab1088deaa58/core/dbt/tracking.py#L468).


Discussion:
> This includes the work to update the internal analytics project to reflect the change in raw event data.

For simplicity, let's kick this out of scope for here, and open a new issue in our `internal-analytics` repo instead to track this as follow-up work: https://github.com/dbt-labs/internal-analytics/issues/1402
> This includes the work to update the internal analytics project to reflect the change in raw event data.

For simplicity, let's kick this out of scope for here, and open a new issue in our `internal-analytics` repo instead to track this as follow-up work: https://github.com/dbt-labs/internal-analytics/issues/1402
@jtcohen6 I was working on this, and I realized we already have the project id as [part of the timing context](https://github.com/dbt-labs/internal-analytics/blob/f5407139dfc970beb960a4dd224874130d132054/models/staging/snowplow/raw/current/dbt/current_snowplow_load_all_timing.sql#L10).  Would it make sense to source it from there as opposed to invoking another snowplow event?
@iknox-fa Good find!

While that should work for the vast majority of cases, I think we want to guarantee that we have `project_id` tracked for 100% of invocations*. My sense is that, a programmatic invocation that [passes in a manifest](https://github.com/dbt-labs/dbt-core/blob/73ff49720021334bd793e55d9d1a198a85ccef5d/core/dbt/cli/main.py#L41) (and therefore _doesn't_ call [`get_full_manifest`](https://github.com/dbt-labs/dbt-core/blob/73ff49720021334bd793e55d9d1a198a85ccef5d/core/dbt/parser/manifest.py#L195)) would never fire that event, and so we'd be missing it for that invocation. That's a pattern we might be pursuing with `dbt-server`.

_*unless the user has opted out of tracking, of course_
Ah Ok that makes sense.  A new event it is!

Resolving patch:
diff --git a/core/dbt/task/base.py b/core/dbt/task/base.py
--- a/core/dbt/task/base.py
+++ b/core/dbt/task/base.py
@@ -77,6 +77,30 @@ def __init__(self, args, config, project=None):
         self.config = config
         self.project = config if isinstance(config, Project) else project
 
+        if dbt.tracking.active_user is not None:
+
+            # N.B. The none checking below is largely due to incomplete projects used in the testing
+            # and to support tasks that don't require a complete project or a complete config (debug, init).
+            project_id = None if self.project is None else self.project.hashed_name()
+            adapter_type = (
+                getattr(self.config.credentials, "type", None)
+                if hasattr(self.config, "credentials")
+                else None
+            )
+            adapter_unique_id = (
+                self.config.credentials.hashed_unique_field()
+                if hasattr(self.config, "credentials")
+                else None
+            )
+
+            dbt.tracking.track_project_id({"project_id": project_id})
+            dbt.tracking.track_adapter_info(
+                {
+                    "adapter_type": adapter_type,
+                    "adapter_unique_id": adapter_unique_id,
+                }
+            )
+
     @classmethod
     def pre_init_hook(cls, args):
         """A hook called before the task is initialized."""
diff --git a/core/dbt/tracking.py b/core/dbt/tracking.py
--- a/core/dbt/tracking.py
+++ b/core/dbt/tracking.py
@@ -30,20 +30,22 @@
 
 COLLECTOR_URL = "fishtownanalytics.sinter-collect.com"
 COLLECTOR_PROTOCOL = "https"
+DBT_INVOCATION_ENV = "DBT_INVOCATION_ENV"
 
-INVOCATION_SPEC = "iglu:com.dbt/invocation/jsonschema/1-0-2"
-PLATFORM_SPEC = "iglu:com.dbt/platform/jsonschema/1-0-0"
-RUN_MODEL_SPEC = "iglu:com.dbt/run_model/jsonschema/1-0-2"
-INVOCATION_ENV_SPEC = "iglu:com.dbt/invocation_env/jsonschema/1-0-0"
-PACKAGE_INSTALL_SPEC = "iglu:com.dbt/package_install/jsonschema/1-0-0"
-RPC_REQUEST_SPEC = "iglu:com.dbt/rpc_request/jsonschema/1-0-1"
+ADAPTER_INFO_SPEC = "iglu:com.dbt/adapter_info/jsonschema/1-0-1"
 DEPRECATION_WARN_SPEC = "iglu:com.dbt/deprecation_warn/jsonschema/1-0-0"
-LOAD_ALL_TIMING_SPEC = "iglu:com.dbt/load_all_timing/jsonschema/1-0-3"
-RESOURCE_COUNTS = "iglu:com.dbt/resource_counts/jsonschema/1-0-0"
 EXPERIMENTAL_PARSER = "iglu:com.dbt/experimental_parser/jsonschema/1-0-0"
+INVOCATION_ENV_SPEC = "iglu:com.dbt/invocation_env/jsonschema/1-0-0"
+INVOCATION_SPEC = "iglu:com.dbt/invocation/jsonschema/1-0-2"
+LOAD_ALL_TIMING_SPEC = "iglu:com.dbt/load_all_timing/jsonschema/1-0-3"
+PACKAGE_INSTALL_SPEC = "iglu:com.dbt/package_install/jsonschema/1-0-0"
 PARTIAL_PARSER = "iglu:com.dbt/partial_parser/jsonschema/1-0-1"
+PLATFORM_SPEC = "iglu:com.dbt/platform/jsonschema/1-0-0"
+PROJECT_ID_SPEC = "iglu:com.dbt/project_id/jsonschema/1-0-1"
+RESOURCE_COUNTS = "iglu:com.dbt/resource_counts/jsonschema/1-0-0"
+RPC_REQUEST_SPEC = "iglu:com.dbt/rpc_request/jsonschema/1-0-1"
 RUNNABLE_TIMING = "iglu:com.dbt/runnable/jsonschema/1-0-0"
-DBT_INVOCATION_ENV = "DBT_INVOCATION_ENV"
+RUN_MODEL_SPEC = "iglu:com.dbt/run_model/jsonschema/1-0-2"
 
 
 class TimeoutEmitter(Emitter):
@@ -210,6 +212,32 @@ def track(user, *args, **kwargs):
             fire_event(SendEventFailure())
 
 
+def track_project_id(options):
+    assert active_user is not None, "Cannot track project_id when active user is None"
+    context = [SelfDescribingJson(PROJECT_ID_SPEC, options)]
+
+    track(
+        active_user,
+        category="dbt",
+        action="project_id",
+        label=get_invocation_id(),
+        context=context,
+    )
+
+
+def track_adapter_info(options):
+    assert active_user is not None, "Cannot track adapter_info when active user is None"
+    context = [SelfDescribingJson(ADAPTER_INFO_SPEC, options)]
+
+    track(
+        active_user,
+        category="dbt",
+        action="adapter_info",
+        label=get_invocation_id(),
+        context=context,
+    )
+
+
 def track_invocation_start(invocation_context):
     data = {"progress": "start", "result_type": None, "result": None}
     data.update(invocation_context)


Test patch: