Repository: meltano/meltano

Issue:
Telemetry project counts inflated if using a custom project_id
Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/3502

Originally created by @aaronsteers on 2022-05-17 18:33:12

---

@edgarrmondragon helpfully pointed me to this block in our [tap-gitlab conversation](https://github.com/MeltanoLabs/tap-gitlab/pull/77#discussion_r875020636):

https://gitlab.com/meltano/meltano/-/blob/4c8cbc0bdb4db0248a378017c3615d88e8cf1878/src/meltano/core/tracking/ga_tracker.py#L84-95

That block apparently fails initialization of project ID _and creates a new one_ if the project ID is not a valid UUID. In my experience, this was getting written back to my project file and then I would continually revert it and it would reset again, etc. In CI, this probably gets a distinct project ID on each run - meaning each run through CI (and each job) looks like a distinct project.

This is very likely creating a significant inflation of our total domain project IDs and I think we should address as ~"urgency::high". (The code was last touched 12+ months ago, or so, so I imaging this has been an issue for quite a while.)

cc @tayloramurphy, @pnadolny13, @edgarrmondragon, @DouweM


Discussion:
[View 16 previous comments from the original issue on GitLab](https://gitlab.com/meltano/meltano/-/issues/3502)

Resolving patch:
diff --git a/src/meltano/core/tracking/ga_tracker.py b/src/meltano/core/tracking/ga_tracker.py
--- a/src/meltano/core/tracking/ga_tracker.py
+++ b/src/meltano/core/tracking/ga_tracker.py
@@ -81,9 +81,16 @@ def load_project_id(self) -> uuid.UUID:
             The project_id.
         """
         project_id_str = self.settings_service.get("project_id")
-        try:
-            project_id = uuid.UUID(project_id_str or "", version=4)
-        except ValueError:
+        if project_id_str:
+            try:
+                # Project ID might already be a UUID
+                project_id = uuid.UUID(project_id_str)
+            except ValueError:
+                # If the project ID is not a UUID, then we hash it, and use the hash to make a UUID
+                project_id = uuid.UUID(
+                    hashlib.sha256(project_id_str.encode()).hexdigest()[::2]
+                )
+        else:
             project_id = uuid.uuid4()
 
             if self.send_anonymous_usage_stats:


Test patch:
diff --git a/.gitlab/ci/test.gitlab-ci.yml b/.gitlab/ci/test.gitlab-ci.yml
--- a/.gitlab/ci/test.gitlab-ci.yml
+++ b/.gitlab/ci/test.gitlab-ci.yml
@@ -27,12 +27,11 @@ lint_python:
       - PYTHON_IMAGE_VERSION: ["3.8"]
   before_script: []
   script:
-    - git remote add meltano https://gitlab.com/meltano/meltano.git || return 0
+    - git remote add meltano https://github.com/meltano/meltano.git || return 0
     - |
-      git fetch meltano $CI_DEFAULT_BRANCH;
-      echo "Comparing to $CI_DEFAULT_BRANCH";
-      REF="meltano/${CI_DEFAULT_BRANCH}";
-    - 'declare FILES=$(git diff --name-only ${CI_COMMIT_SHA} $REF | grep "\.py$")'
+      git fetch meltano main;
+      echo "Comparing to main";
+    - 'declare FILES=$(git diff --name-only ${CI_COMMIT_SHA} meltano/main | grep "\.py$")'
     - |
       if [ -n "${FILES}" ]; then
         echo "Changed files are\n$FILES";