Repository: dbt-labs/dbt-core

Issue:
[Feature] An explicitly specified prerelease package should install without requiring install-prerelease
### Is there an existing feature request for this?

- [X] I have searched the existing issues

### Describe the Feature

I opened #4224 because I tried to install a prerelease version of dbt utils without the `install-prerelease: True` setting: 
```
packages:
  - package: dbt-labs/dbt_utils
    version: 0.7.4-b1 
```

Requesting a prerelease package by name should be enough to trigger its installation.

### Describe alternatives you've considered

_No response_

### Who will this benefit?

Test drivers of beta packages 

### Are you interested in contributing this feature?

Only if no one else beats me to it

### Anything else?

A [potential extra for experts detailed here](https://github.com/dbt-labs/dbt-core/issues/4224#issuecomment-962969664) is to also add a `--pre` subcommand to `dbt deps` which behaves like `pip install x --pre` and installs prerelease versions of all first-order packages (i.e. not if you're getting dbt-utils by dependency instead of a direct reference). 


Discussion:
Yes, we should absolutely do this!

I was afraid the logic here would be pretty gnarly, but after a quick dive in, it's much simpler than I thought:

https://github.com/dbt-labs/dbt-core/blob/22416980d19888d7bbbcfcda639c16d37a80b948/core/dbt/deps/registry.py#L130-L133

Becomes:

```python
        # if any prelease version is explicitly specified in any way,
        # assume we're good to use prereleases for this package
        prerelease_version_specified = any(
            bool(version.prerelease) for version in self.versions
        )
        installable = semver.filter_installable(
            available,
            self.install_prerelease or prerelease_version_specified
        )
```

This would have the effect of supporting both of the following:
```yml
packages:
  - package: dbt-labs/dbt_utils
    version: 0.7.4-b1 
```
```yml
packages:
  - package: dbt-labs/dbt_utils
    version: [">=0.7.4-b1", "<0.7.5"]
```
```
$ dbt deps
20:07:06 | [ info  ] | Running with dbt=1.0.0-rc1
20:07:06 | [ info  ] | Installing dbt-labs/dbt_utils@0.7.4
20:07:06 | [ info  ] |   Installed from version 0.7.4
20:07:06 | [ info  ] |   Up to date!
```

I'd welcome a PR for this!

---

A larger question: to what extent should we try to follow [PEP 404](https://www.python.org/dev/peps/pep-0440/#handling-of-pre-releases) to the letter? (I stumbled across some interesting related conversation here: https://github.com/pypa/pip/issues/5503).

Namely:
- If a user explicitly specifies a prerelease version for a package, are they opting into that prerelease version? YES
- If a user includes _any_ prerelease marker in their version specification for a package, are they opting into prereleases for that package? I say YES
- If a prerelease version is the only valid and available version, should dbt install it? This feels a bit more controversial; I think yes, but it's worth discussing more. It would require a modification to the logic in `filter_installable`, to prepare both `installable` and `installable_with_prereleases`, and hand back the latter if the former is unavailable.

Resolving patch:
diff --git a/core/dbt/deps/registry.py b/core/dbt/deps/registry.py
--- a/core/dbt/deps/registry.py
+++ b/core/dbt/deps/registry.py
@@ -127,9 +127,12 @@ def resolved(self) -> RegistryPinnedPackage:
             raise DependencyException(new_msg) from e
 
         available = registry.get_available_versions(self.package)
+        prerelease_version_specified = any(
+            bool(version.prerelease) for version in self.versions
+        )
         installable = semver.filter_installable(
             available,
-            self.install_prerelease
+            self.install_prerelease or prerelease_version_specified
         )
         available_latest = installable[-1]
 


Test patch:
diff --git a/test/unit/test_deps.py b/test/unit/test_deps.py
--- a/test/unit/test_deps.py
+++ b/test/unit/test_deps.py
@@ -472,6 +472,36 @@ def test_get_version_latest_prelease_false(self):
         self.assertEqual(c_pinned.get_version_latest(), '0.1.3')
         self.assertEqual(c_pinned.source_type(), 'hub')
 
+    def test_get_version_prerelease_explicitly_requested(self):
+        a_contract = RegistryPackage(
+            package='dbt-labs-test/a',
+            version='0.1.4a1',
+            install_prerelease=None
+        )
+
+        a = RegistryUnpinnedPackage.from_contract(a_contract)
+
+        self.assertEqual(a.package, 'dbt-labs-test/a')
+        self.assertEqual(
+            a.versions,
+            [
+                VersionSpecifier(
+                    build=None,
+                    major='0',
+                    matcher='=',
+                    minor='1',
+                    patch='4',
+                    prerelease='a1',
+                ),
+            ]
+        )
+
+        a_pinned = a.resolved()
+        self.assertEqual(a_pinned.package, 'dbt-labs-test/a')
+        self.assertEqual(a_pinned.version, '0.1.4a1')
+        self.assertEqual(a_pinned.get_version_latest(), '0.1.4a1')
+        self.assertEqual(a_pinned.source_type(), 'hub')
+
 class MockRegistry:
     def __init__(self, packages):
         self.packages = packages