Repository: DataDog/guarddog (python)

Issue:
Guarddog hanging when it receices 404 from PyPi 
We have a bunch of internal libraries that are not uploaded to pypi (and never will be) but rather included in a form

```
${package} @ file:///tmp/${package}-${version}.tar.gz
```

whenever guarddog finds such a package it tries to load it form pypi and then the whole process hangs and doesn't complete execution.  

Pull request:
Ignore packages that don't exist in PyPI (closes #78)
When scanning requirements.txt files, we shouldn't crash on packages that don't exist on PyPI. See also #78 

Patch:
diff --git a/.github/workflows/pypi-release.yml b/.github/workflows/pypi-release.yml
new file mode 100644
index 0000000..c5eb580
--- /dev/null
+++ b/.github/workflows/pypi-release.yml
@@ -0,0 +1,35 @@
+name: Release PyPI package
+
+on:
+  push:
+    tags:
+      - "v*"
+
+permissions:
+  contents: read
+
+jobs:
+  build-and-publish:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v2.5.0
+        with:
+          fetch-depth: 0
+
+      - name: 'Set up Python 3.10'
+        uses: actions/setup-python@v3
+        with:
+          python-version: "3.10"
+
+      - name: Install pypa/build
+        run: python -m pip install build --user
+
+      - name: Build a binary wheel and a source tarball
+        run: python -m build --sdist --wheel --outdir dist/ .
+
+      - name: Publish distribution 📦 to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1.5.1
+        with:
+          password: ${{ secrets.PYPI_PUBLISH_TOKEN }}
+        
\ No newline at end of file
diff --git a/Makefile b/Makefile
index 1e42be0..d01532c 100644
--- a/Makefile
+++ b/Makefile
@@ -1,9 +1,12 @@
-.PHONY: test test-semgrep-rules test-metadata-rules
+.PHONY: test test-semgrep-rules test-metadata-rules test-core
 
-test: test-semgrep-rules test-metadata-rules
+test: test-semgrep-rules test-metadata-rules test-core
 
 test-semgrep-rules:
 	semgrep --metrics off --quiet --test --config guarddog/analyzer/sourcecode tests/analyzer/sourcecode
 
 test-metadata-rules:
-	python -m pytest
+	python -m pytest tests/analyzer/metadata
+
+test-core:
+	python -m pytest tests/core
diff --git a/README.md b/README.md
index d5ab514..0a953fa 100644
--- a/README.md
+++ b/README.md
@@ -13,7 +13,7 @@ GuardDog can be used to scan local or remote PyPI packages using any of the avai
 ### Installation
 
 ```sh
-pip install git+https://github.com/DataDog/guarddog.git
+pip install guarddog
 ```
 
 Or use the Docker image:
diff --git a/guarddog/scanners/project_scanner.py b/guarddog/scanners/project_scanner.py
index d00ddcd..823a0d5 100644
--- a/guarddog/scanners/project_scanner.py
+++ b/guarddog/scanners/project_scanner.py
@@ -100,13 +100,15 @@ class RequirementsScanner(Scanner):
         try:
             for requirement in pkg_resources.parse_requirements(sanitized_requirements):
                 valid_versions = None
+                project_exists_on_pypi = True
                 for spec in requirement.specs:
                     qualifier, version = spec
 
                     try:
                         available_versions = versions(requirement.project_name)
                     except Exception:
-                        sys.stderr.write(f"Package {requirement.project_name} not on PyPI")
+                        sys.stderr.write(f"Package {requirement.project_name} not on PyPI\n")
+                        project_exists_on_pypi = False
                         continue
 
                     used_versions = None
@@ -142,7 +144,8 @@ class RequirementsScanner(Scanner):
                     else:
                         valid_versions = valid_versions & used_versions
 
-                dependencies[requirement.project_name] = valid_versions
+                if project_exists_on_pypi:
+                    dependencies[requirement.project_name] = valid_versions
         except Exception as e:
             sys.stderr.write(f"Received error {str(e)}")
 
diff --git a/pyproject.toml b/pyproject.toml
index ea6cf3a..2ff1ef2 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -59,5 +59,6 @@ filterwarnings = [
 ]
 
 testpaths = [
-    "tests/analyzer/metadata"
+    "tests/analyzer/metadata",
+    "tests/core"
 ]


Test patch:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 874636e..3411359 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -35,6 +35,8 @@ jobs:
       run: make test-semgrep-rules
     - name: Python unit tests
       run: make test-metadata-rules
+    - name: Core unit tests
+      run: make test-core
 
   docker-build:
     runs-on: ubuntu-latest
diff --git a/tests/core/test_requirements_scanner.py b/tests/core/test_requirements_scanner.py
new file mode 100644
index 0000000..e650ffb
--- /dev/null
+++ b/tests/core/test_requirements_scanner.py
@@ -0,0 +1,8 @@
+from guarddog.scanners.project_scanner import RequirementsScanner
+
+
+# Regression test for https://github.com/DataDog/guarddog/issues/78
+def test_requirements_scanner_on_non_existing_package():
+    scanner = RequirementsScanner()
+    result = scanner.parse_requirements(["not-a-real-package==1.0.0"])
+    assert "not-a-real-package" not in result