Repository: scala-steward-org/scala-steward
Pull request #2933 (2023-01-18 14:16:14)

Extract dependencies from scala-cli via sbt export
This implements the idea sketched in https://github.com/scala-steward-org/scala-steward/issues/2686#issuecomment-1386629936. `ScalaCliAlg` extracts dependencies and resolvers from a Scala CLI "project" via an export to a sbt build and then leveraging the already existing `SbtAlg` for the actual data extraction.

These two PRs demonstrate that this works:
* https://github.com/scala-steward-org/test-repo-1/pull/35/files
* https://github.com/scala-steward-org/test-repo-1/pull/36/files

Note that the base branch of these PRs only contain a single [`script.scala`](https://github.com/scala-steward-org/test-repo-1/blob/cd20385d988ff46b52d72bfdf7f235f61cf4545f/script.scala) file. There are no other files.

codecov[bot]: # [Codecov](https://codecov.io/gh/scala-steward-org/scala-steward/pull/2933?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org) Report
Base: **90.57**% // Head: **90.27**% // Decreases project coverage by **`-0.30%`** :warning:
> Coverage data is based on head [(`730e474`)](https://codecov.io/gh/scala-steward-org/scala-steward/pull/2933?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org) compared to base [(`71831fe`)](https://codecov.io/gh/scala-steward-org/scala-steward/commit/71831fe822bd01fc2be2e01939d8f991165a4b60?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org).
> Patch coverage: 18.75% of modified lines in pull request are covered.

> :exclamation: Current head 730e474 differs from pull request most recent head 62d8f2b. Consider uploading reports for the commit 62d8f2b to get more accurate results

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #2933      +/-   ##
==========================================
- Coverage   90.57%   90.27%   -0.30%     
==========================================
  Files         154      155       +1     
  Lines        3044     3085      +41     
  Branches      196      204       +8     
==========================================
+ Hits         2757     2785      +28     
- Misses        287      300      +13     
```


| [Impacted Files](https://codecov.io/gh/scala-steward-org/scala-steward/pull/2933?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org) | Coverage Δ | |
|---|---|---|
| [...asteward/core/buildtool/scalacli/ScalaCliAlg.scala](https://codecov.io/gh/scala-steward-org/scala-steward/pull/2933?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org#diff-bW9kdWxlcy9jb3JlL3NyYy9tYWluL3NjYWxhL29yZy9zY2FsYXN0ZXdhcmQvY29yZS9idWlsZHRvb2wvc2NhbGFjbGkvU2NhbGFDbGlBbGcuc2NhbGE=) | `7.14% <7.14%> (ø)` | |
| [...la/org/scalasteward/core/application/Context.scala](https://codecov.io/gh/scala-steward-org/scala-steward/pull/2933?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org#diff-bW9kdWxlcy9jb3JlL3NyYy9tYWluL3NjYWxhL29yZy9zY2FsYXN0ZXdhcmQvY29yZS9hcHBsaWNhdGlvbi9Db250ZXh0LnNjYWxh) | `75.32% <100.00%> (+0.32%)` | :arrow_up: |
| [...lasteward/core/buildtool/BuildToolDispatcher.scala](https://codecov.io/gh/scala-steward-org/scala-steward/pull/2933?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org#diff-bW9kdWxlcy9jb3JlL3NyYy9tYWluL3NjYWxhL29yZy9zY2FsYXN0ZXdhcmQvY29yZS9idWlsZHRvb2wvQnVpbGRUb29sRGlzcGF0Y2hlci5zY2FsYQ==) | `100.00% <100.00%> (ø)` | |
| [...a/org/scalasteward/core/edit/update/Selector.scala](https://codecov.io/gh/scala-steward-org/scala-steward/pull/2933?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org#diff-bW9kdWxlcy9jb3JlL3NyYy9tYWluL3NjYWxhL29yZy9zY2FsYXN0ZXdhcmQvY29yZS9lZGl0L3VwZGF0ZS9TZWxlY3Rvci5zY2FsYQ==) | `100.00% <0.00%> (ø)` | |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/scala-steward-org/scala-steward/pull/2933?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=scala-steward-org).

fthomas: Btw, this integration could be nicer and less hacky with a `dump` command as described in https://github.com/VirtusLab/scala-cli/discussions/1005.

Patch:
diff --git a/.scalafmt.conf b/.scalafmt.conf
index 66301fb1..e158356e 100644
--- a/.scalafmt.conf
+++ b/.scalafmt.conf
@@ -1,4 +1,4 @@
-version = 3.6.1
+version = 3.7.0
 runner.dialect = scala213
 assumeStandardLibraryStripMargin = true
 align.openParenCallSite = false
diff --git a/build.sbt b/build.sbt
index 6336788c..0c313721 100644
--- a/build.sbt
+++ b/build.sbt
@@ -348,6 +348,7 @@ lazy val dockerSettings = Def.settings(
       Cmd("RUN", "apk --no-cache add bash git ca-certificates curl maven openssh nodejs npm"),
       Cmd("RUN", s"wget $sbtUrl && tar -xf $sbtTgz && rm -f $sbtTgz"),
       Cmd("RUN", s"curl -L $millUrl > $millBin && chmod +x $millBin"),
+      Cmd("RUN", "curl -sSLf https://virtuslab.github.io/scala-cli-packages/scala-setup.sh | sh"),
       Cmd("RUN", s"curl -L https://git.io/coursier-cli > $coursierBin && chmod +x $coursierBin"),
       Cmd("RUN", s"$coursierBin install --install-dir $binDir scalafix scalafmt"),
       Cmd("RUN", "npm install --global yarn")
diff --git a/modules/core/src/main/scala/org/scalasteward/core/application/Context.scala b/modules/core/src/main/scala/org/scalasteward/core/application/Context.scala
index 381f0ad1..710df09d 100644
--- a/modules/core/src/main/scala/org/scalasteward/core/application/Context.scala
+++ b/modules/core/src/main/scala/org/scalasteward/core/application/Context.scala
@@ -30,6 +30,7 @@ import org.scalasteward.core.buildtool.BuildToolDispatcher
 import org.scalasteward.core.buildtool.maven.MavenAlg
 import org.scalasteward.core.buildtool.mill.MillAlg
 import org.scalasteward.core.buildtool.sbt.SbtAlg
+import org.scalasteward.core.buildtool.scalacli.ScalaCliAlg
 import org.scalasteward.core.client.ClientConfiguration
 import org.scalasteward.core.coursier.{CoursierAlg, VersionsCache}
 import org.scalasteward.core.data.Repo
@@ -75,6 +76,7 @@ final class Context[F[_]](implicit
     val repoCacheAlg: RepoCacheAlg[F],
     val repoConfigAlg: RepoConfigAlg[F],
     val sbtAlg: SbtAlg[F],
+    val scalaCliAlg: ScalaCliAlg[F],
     val scalafixMigrationsFinder: ScalafixMigrationsFinder,
     val scalafixMigrationsLoader: ScalafixMigrationsLoader[F],
     val scalafmtAlg: ScalafmtAlg[F],
@@ -221,6 +223,7 @@ object Context {
       implicit val updateAlg: UpdateAlg[F] = new UpdateAlg[F]
       implicit val mavenAlg: MavenAlg[F] = new MavenAlg[F](config)
       implicit val sbtAlg: SbtAlg[F] = new SbtAlg[F](config)
+      implicit val scalaCliAlg: ScalaCliAlg[F] = new ScalaCliAlg[F]
       implicit val millAlg: MillAlg[F] = new MillAlg[F]
       implicit val buildToolDispatcher: BuildToolDispatcher[F] = new BuildToolDispatcher[F]
       implicit val refreshErrorAlg: RefreshErrorAlg[F] =
diff --git a/modules/core/src/main/scala/org/scalasteward/core/buildtool/BuildToolDispatcher.scala b/modules/core/src/main/scala/org/scalasteward/core/buildtool/BuildToolDispatcher.scala
index bc3a0d0a..8a5da8d6 100644
--- a/modules/core/src/main/scala/org/scalasteward/core/buildtool/BuildToolDispatcher.scala
+++ b/modules/core/src/main/scala/org/scalasteward/core/buildtool/BuildToolDispatcher.scala
@@ -21,6 +21,7 @@ import cats.syntax.all._
 import org.scalasteward.core.buildtool.maven.MavenAlg
 import org.scalasteward.core.buildtool.mill.MillAlg
 import org.scalasteward.core.buildtool.sbt.SbtAlg
+import org.scalasteward.core.buildtool.scalacli.ScalaCliAlg
 import org.scalasteward.core.data.{Repo, Scope}
 import org.scalasteward.core.edit.scalafix.ScalafixMigration
 import org.scalasteward.core.repoconfig.RepoConfig
@@ -32,6 +33,7 @@ final class BuildToolDispatcher[F[_]](implicit
     mavenAlg: MavenAlg[F],
     millAlg: MillAlg[F],
     sbtAlg: SbtAlg[F],
+    scalaCliAlg: ScalaCliAlg[F],
     scalafmtAlg: ScalafmtAlg[F],
     F: Monad[F]
 ) {
@@ -54,7 +56,7 @@ final class BuildToolDispatcher[F[_]](implicit
   private def getBuildRoots(repo: Repo, repoConfig: RepoConfig): List[BuildRoot] =
     repoConfig.buildRootsOrDefault.map(buildRootCfg => BuildRoot(repo, buildRootCfg.relativePath))
 
-  private val allBuildTools = List(mavenAlg, millAlg, sbtAlg)
+  private val allBuildTools = List(mavenAlg, millAlg, sbtAlg, scalaCliAlg)
   private val fallbackBuildTool = List(sbtAlg)
 
   private def findBuildTools(buildRoot: BuildRoot): F[(BuildRoot, List[BuildToolAlg[F]])] =
diff --git a/modules/core/src/main/scala/org/scalasteward/core/buildtool/maven/MavenAlg.scala b/modules/core/src/main/scala/org/scalasteward/core/buildtool/maven/MavenAlg.scala
index c74c54a5..678c3531 100644
--- a/modules/core/src/main/scala/org/scalasteward/core/buildtool/maven/MavenAlg.scala
+++ b/modules/core/src/main/scala/org/scalasteward/core/buildtool/maven/MavenAlg.scala
@@ -58,7 +58,7 @@ final class MavenAlg[F[_]](config: Config)(implicit
 
   override def runMigration(buildRoot: BuildRoot, migration: ScalafixMigration): F[Unit] =
     logger.warn(
-      "Scalafix migrations are currently not supported in Maven projects, see https://github.com/scala-steward-org/scala-steward/issues/2839 for details"
+      s"Scalafix migrations are currently not supported in $name projects, see https://github.com/scala-steward-org/scala-steward/issues/2839 for details"
     )
 
   private def exec(command: Nel[String], repoDir: File): F[List[String]] =
diff --git a/modules/core/src/main/scala/org/scalasteward/core/buildtool/mill/MillAlg.scala b/modules/core/src/main/scala/org/scalasteward/core/buildtool/mill/MillAlg.scala
index 199c12f2..94cd015e 100644
--- a/modules/core/src/main/scala/org/scalasteward/core/buildtool/mill/MillAlg.scala
+++ b/modules/core/src/main/scala/org/scalasteward/core/buildtool/mill/MillAlg.scala
@@ -21,7 +21,6 @@ import cats.effect.MonadCancelThrow
 import cats.syntax.all._
 import org.scalasteward.core.buildtool.mill.MillAlg._
 import org.scalasteward.core.buildtool.{BuildRoot, BuildToolAlg}
-import org.scalasteward.core.data.Scope.Dependencies
 import org.scalasteward.core.data._
 import org.scalasteward.core.edit.scalafix.ScalafixMigration
 import org.scalasteward.core.io.{FileAlg, ProcessAlg, WorkspaceAlg}
@@ -42,7 +41,7 @@ final class MillAlg[F[_]](implicit
       .buildRootDir(buildRoot)
       .flatMap(buildRootDir => fileAlg.isRegularFile(buildRootDir / "build.sc"))
 
-  override def getDependencies(buildRoot: BuildRoot): F[List[Dependencies]] =
+  override def getDependencies(buildRoot: BuildRoot): F[List[Scope.Dependencies]] =
     for {
       buildRootDir <- workspaceAlg.buildRootDir(buildRoot)
       predef = buildRootDir / "scala-steward.sc"
@@ -66,7 +65,7 @@ final class MillAlg[F[_]](implicit
 
   override def runMigration(buildRoot: BuildRoot, migration: ScalafixMigration): F[Unit] =
     logger.warn(
-      "Scalafix migrations are currently not supported in Mill projects, see https://github.com/scala-steward-org/scala-steward/issues/2838 for details"
+      s"Scalafix migrations are currently not supported in $name projects, see https://github.com/scala-steward-org/scala-steward/issues/2838 for details"
     )
 
   private def getMillVersion(buildRootDir: File): F[Option[Version]] =
diff --git a/modules/core/src/main/scala/org/scalasteward/core/buildtool/scalacli/ScalaCliAlg.scala b/modules/core/src/main/scala/org/scalasteward/core/buildtool/scalacli/ScalaCliAlg.scala
new file mode 100644
index 00000000..6b55e40c
--- /dev/null
+++ b/modules/core/src/main/scala/org/scalasteward/core/buildtool/scalacli/ScalaCliAlg.scala
@@ -0,0 +1,64 @@
+/*
+ * Copyright 2018-2023 Scala Steward contributors
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.scalasteward.core.buildtool.scalacli
+
+import cats.Monad
+import cats.syntax.all._
+import org.scalasteward.core.buildtool.sbt.SbtAlg
+import org.scalasteward.core.buildtool.{BuildRoot, BuildToolAlg}
+import org.scalasteward.core.data.Scope
+import org.scalasteward.core.edit.scalafix.ScalafixMigration
+import org.scalasteward.core.git.GitAlg
+import org.scalasteward.core.io.{FileAlg, ProcessAlg, WorkspaceAlg}
+import org.scalasteward.core.util.Nel
+import org.typelevel.log4cats.Logger
+
+final class ScalaCliAlg[F[_]](implicit
+    fileAlg: FileAlg[F],
+    gitAlg: GitAlg[F],
+    logger: Logger[F],
+    processAlg: ProcessAlg[F],
+    sbtAlg: SbtAlg[F],
+    workspaceAlg: WorkspaceAlg[F],
+    F: Monad[F]
+) extends BuildToolAlg[F] {
+  override def name: String = "Scala CLI"
+
+  override def containsBuild(buildRoot: BuildRoot): F[Boolean] = {
+    val buildRootPath = buildRoot.relativePath.dropWhile(Set('.', '/'))
+    gitAlg
+      .findFilesContaining(buildRoot.repo, "//> using lib ")
+      .map(_.exists(_.startsWith(buildRootPath)))
+  }
+
+  override def getDependencies(buildRoot: BuildRoot): F[List[Scope.Dependencies]] =
+    for {
+      buildRootDir <- workspaceAlg.buildRootDir(buildRoot)
+      exportDir = "tmp-sbt-build-for-scala-steward"
+      exportCmd =
+        Nel.of("scala-cli", "export", "--sbt", "--output", exportDir, buildRootDir.pathAsString)
+      _ <- processAlg.execSandboxed(exportCmd, buildRootDir)
+      exportBuildRoot = buildRoot.copy(relativePath = buildRoot.relativePath + s"/$exportDir")
+      dependencies <- sbtAlg.getDependencies(exportBuildRoot)
+      _ <- fileAlg.deleteForce(buildRootDir / exportDir)
+    } yield dependencies
+
+  override def runMigration(buildRoot: BuildRoot, migration: ScalafixMigration): F[Unit] =
+    logger.warn(
+      s"Scalafix migrations are currently not supported in $name projects, see https://github.com/VirtusLab/scala-cli/issues/647 for details"
+    )
+}
diff --git a/repos.md b/repos.md
index ebb9d87a..39136251 100644
--- a/repos.md
+++ b/repos.md
@@ -10,4 +10,5 @@ All lines that do not start with a hyphen and space are ignored.
 - scala-steward-org/sbt-plugin
 - scala-steward-org/scala-steward
 - scala-steward-org/test-repo-1
+- scala-steward-org/test-repo-1:scala-cli-test
 - scala-steward-org/test-repo-2


Test patch:
diff --git a/modules/core/src/test/scala/org/scalasteward/core/buildtool/BuildToolDispatcherTest.scala b/modules/core/src/test/scala/org/scalasteward/core/buildtool/BuildToolDispatcherTest.scala
index 76c4f66c..a96fe2cc 100644
--- a/modules/core/src/test/scala/org/scalasteward/core/buildtool/BuildToolDispatcherTest.scala
+++ b/modules/core/src/test/scala/org/scalasteward/core/buildtool/BuildToolDispatcherTest.scala
@@ -4,6 +4,7 @@ import cats.effect.unsafe.implicits.global
 import munit.FunSuite
 import org.scalasteward.core.buildtool.sbt.command._
 import org.scalasteward.core.data._
+import org.scalasteward.core.mock.MockConfig.gitCmd
 import org.scalasteward.core.mock.MockContext.context._
 import org.scalasteward.core.mock.MockState
 import org.scalasteward.core.mock.MockState.TraceEntry.{Cmd, Log}
@@ -33,9 +34,25 @@ class BuildToolDispatcherTest extends FunSuite {
         Cmd("test", "-f", s"$repoDir/pom.xml"),
         Cmd("test", "-f", s"$repoDir/build.sc"),
         Cmd("test", "-f", s"$repoDir/build.sbt"),
+        Cmd(
+          gitCmd(repoDir),
+          "grep",
+          "-I",
+          "--fixed-strings",
+          "--files-with-matches",
+          "//> using lib "
+        ),
         Cmd("test", "-f", s"$repoDir/mvn-build/pom.xml"),
         Cmd("test", "-f", s"$repoDir/mvn-build/build.sc"),
         Cmd("test", "-f", s"$repoDir/mvn-build/build.sbt"),
+        Cmd(
+          gitCmd(repoDir),
+          "grep",
+          "-I",
+          "--fixed-strings",
+          "--files-with-matches",
+          "//> using lib "
+        ),
         Log("Get dependencies in . from sbt"),
         Cmd("read", s"$repoDir/project/build.properties"),
         Cmd("read", "classpath:StewardPlugin_1_0_0.scala"),
diff --git a/modules/core/src/test/scala/org/scalasteward/core/buildtool/scalacli/ScalaCliAlgTest.scala b/modules/core/src/test/scala/org/scalasteward/core/buildtool/scalacli/ScalaCliAlgTest.scala
new file mode 100644
index 00000000..85fcbac6
--- /dev/null
+++ b/modules/core/src/test/scala/org/scalasteward/core/buildtool/scalacli/ScalaCliAlgTest.scala
@@ -0,0 +1,75 @@
+package org.scalasteward.core.buildtool.scalacli
+
+import munit.CatsEffectSuite
+import org.scalasteward.core.buildtool.BuildRoot
+import org.scalasteward.core.buildtool.sbt.command._
+import org.scalasteward.core.data.{GroupId, Repo, Version}
+import org.scalasteward.core.edit.scalafix.ScalafixMigration
+import org.scalasteward.core.mock.MockContext.context._
+import org.scalasteward.core.mock.MockState
+import org.scalasteward.core.mock.MockState.TraceEntry.{Cmd, Log}
+import org.scalasteward.core.util.Nel
+
+class ScalaCliAlgTest extends CatsEffectSuite {
+  test("getDependencies") {
+    val repo = Repo("user", "repo")
+    val buildRoot = BuildRoot(repo, ".")
+    val repoDir = workspaceAlg.repoDir(repo).unsafeRunSync()
+    val sbtBuildDir = repoDir / "tmp-sbt-build-for-scala-steward"
+
+    val obtained = scalaCliAlg.getDependencies(buildRoot).runS(MockState.empty)
+    val expected = MockState.empty.copy(trace =
+      Vector(
+        Cmd(
+          repoDir.toString,
+          "firejail",
+          "--quiet",
+          s"--whitelist=$repoDir",
+          "--env=VAR1=val1",
+          "--env=VAR2=val2",
+          "scala-cli",
+          "export",
+          "--sbt",
+          "--output",
+          "tmp-sbt-build-for-scala-steward",
+          repoDir.toString
+        ),
+        Cmd("read", s"$sbtBuildDir/project/build.properties"),
+        Cmd("read", "classpath:StewardPlugin_1_3_11.scala"),
+        Cmd("write", s"$sbtBuildDir/project/scala-steward-StewardPlugin_1_3_11.scala"),
+        Cmd("write", s"$sbtBuildDir/project/project/scala-steward-StewardPlugin_1_3_11.scala"),
+        Cmd(
+          sbtBuildDir.toString,
+          "firejail",
+          "--quiet",
+          s"--whitelist=$sbtBuildDir",
+          "--env=VAR1=val1",
+          "--env=VAR2=val2",
+          "sbt",
+          "-Dsbt.color=false",
+          "-Dsbt.log.noformat=true",
+          "-Dsbt.supershell=false",
+          s";$crossStewardDependencies;$reloadPlugins;$stewardDependencies"
+        ),
+        Cmd("rm", "-rf", s"$sbtBuildDir/project/project/scala-steward-StewardPlugin_1_3_11.scala"),
+        Cmd("rm", "-rf", s"$sbtBuildDir/project/scala-steward-StewardPlugin_1_3_11.scala"),
+        Cmd("rm", "-rf", s"$sbtBuildDir")
+      )
+    )
+
+    assertIO(obtained, expected)
+  }
+
+  test("runMigration") {
+    val repo = Repo("user", "repo")
+    val buildRoot = BuildRoot(repo, ".")
+    val migration = ScalafixMigration(
+      GroupId("co.fs2"),
+      Nel.of("fs2-core"),
+      Version("1.0.0"),
+      Nel.of("github:functional-streams-for-scala/fs2/v1?sha=v1.0.5")
+    )
+    val obtained = scalaCliAlg.runMigration(buildRoot, migration).runS(MockState.empty)
+    assertIO(obtained.map(_.trace.collect { case Log(_) => () }.size), 1)
+  }
+}