Repository: kestra-io/kestra
Issue: Tune on Flyway migration logs
Labels: technical-issue

### Feature description

We use Flyway to automatically migrate our database schema.
As Flyway logs a lot, we disable most of its logs.

The issue is that when a migration is long, there is nothing in the logs that indicates that Flyway is running, as we also disable Micronaut starting logs and Kestra didn't log nothing before the migration.

We may consider turning on some logs (Flyway or Micronaut) or adding a log at startup so users often think that something is broken when they just need to wait for the migration to happen.

Issue comments:
By setting `<logger name="org.flywaydb.core.internal.command.DbMigrate" level="INFO" />` we would have Flyway migration logs.

For ex when no migration:
```
2024-03-12 10:09:12,338 INFO  main         o.f.core.internal.command.DbMigrate Current version of schema "public": 1.18
2024-03-12 10:09:12,342 INFO  main         o.f.core.internal.command.DbMigrate Schema "public" is up to date. No migration necessary.
2024-03-12 10:09:12,338 WARN  main         o.f.core.internal.command.DbMigrate outOfOrder mode is active. Migration of schema "public" may not be reproducible.
```

And when a migration happens:
```
2024-03-12 10:19:29,359 INFO  main         o.f.core.internal.command.DbMigrate Current version of schema "public": 1.18
2024-03-12 10:19:29,378 INFO  main         o.f.core.internal.command.DbMigrate Migrating schema "public" to version "1.19 - test"
2024-03-12 10:19:29,360 WARN  main         o.f.core.internal.command.DbMigrate outOfOrder mode is active. Migration of schema "public" may not be reproducible.
2024-03-12 10:19:29,425 INFO  main         o.f.core.internal.command.DbMigrate Successfully applied 1 migration to schema "public", now at version v1.19 (execution time 00:00.008s)
```

The issue is that, as we use out-of-order migration, if we do this we would have a WARN log from Flyway.
To suppress it we can filter it in logback it would incur a small cost.


Pull request review comments:

Patch that resolved the issue:
diff --git a/build.gradle b/build.gradle
index 1db28dc587..74a18a6630 100644
--- a/build.gradle
+++ b/build.gradle
@@ -126,6 +126,7 @@ allprojects {
         // logs
         implementation "org.slf4j:slf4j-api"
         implementation "ch.qos.logback:logback-classic"
+        implementation "org.codehaus.janino:janino:3.1.12"
         implementation group: 'org.apache.logging.log4j', name: 'log4j-to-slf4j', version: '2.23.0'
         implementation group: 'org.slf4j', name: 'jul-to-slf4j', version: slf4jVersion
         implementation group: 'org.slf4j', name: 'jcl-over-slf4j', version: slf4jVersion
diff --git a/core/src/main/resources/logback/base.xml b/core/src/main/resources/logback/base.xml
index 4b27bcfb48..04f044970b 100644
--- a/core/src/main/resources/logback/base.xml
+++ b/core/src/main/resources/logback/base.xml
@@ -32,7 +32,11 @@
     <!-- Elastic deprecation warning that is not compatible with OpenSearch, we must disable all warnings ... -->
     <logger name="org.opensearch.client.RestClient" level="ERROR" />
 
-    <!-- Flyway log warnings for out-of-order or when using IF NOT EXISTS -->
-    <logger name="org.flywaydb.core.internal.command.DbMigrate" level="ERROR" />
+    <!--
+        We enable INFO for DbMigrate otherwise there is nothing displayed on the console when a migration is ongoing,
+        which can be make thinking Kestra is not starting when the migration is long.
+    -->
+    <logger name="org.flywaydb.core.internal.command.DbMigrate" level="INFO" />
+    <!-- Flyway log when using IF NOT EXISTS -->
     <logger name="org.flywaydb.core.internal.sqlscript.DefaultSqlScriptExecutor" level="ERROR" />
 </included>
diff --git a/core/src/main/resources/logback/ecs.xml b/core/src/main/resources/logback/ecs.xml
index 0d7c297601..2213f719ff 100644
--- a/core/src/main/resources/logback/ecs.xml
+++ b/core/src/main/resources/logback/ecs.xml
@@ -20,6 +20,13 @@
         <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
             <level>WARN</level>
         </filter>
+        <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
+            <evaluator>
+                <expression>return message.contains("outOfOrder mode is active. Migration of schema");</expression>
+            </evaluator>
+            <OnMismatch>NEUTRAL</OnMismatch>
+            <OnMatch>DENY</OnMatch>
+        </filter>
         <encoder class="co.elastic.logging.logback.EcsEncoder" />
     </appender>
 </included>
diff --git a/core/src/main/resources/logback/gcp.xml b/core/src/main/resources/logback/gcp.xml
index e286e1345a..892eed7856 100644
--- a/core/src/main/resources/logback/gcp.xml
+++ b/core/src/main/resources/logback/gcp.xml
@@ -23,6 +23,13 @@
         <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
             <level>WARN</level>
         </filter>
+        <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
+            <evaluator>
+                <expression>return message.contains("outOfOrder mode is active. Migration of schema");</expression>
+            </evaluator>
+            <OnMismatch>NEUTRAL</OnMismatch>
+            <OnMatch>DENY</OnMatch>
+        </filter>
         <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
             <layout class="io.kestra.cli.logger.StackdriverJsonLayout">
             </layout>
diff --git a/core/src/main/resources/logback/text.xml b/core/src/main/resources/logback/text.xml
index a24ade89e1..94efdaee06 100644
--- a/core/src/main/resources/logback/text.xml
+++ b/core/src/main/resources/logback/text.xml
@@ -22,6 +22,13 @@
         <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
             <level>WARN</level>
         </filter>
+        <filter class="ch.qos.logback.core.filter.EvaluatorFilter">
+            <evaluator>
+                <expression>return message.contains("outOfOrder mode is active. Migration of schema");</expression>
+            </evaluator>
+            <OnMismatch>NEUTRAL</OnMismatch>
+            <OnMatch>DENY</OnMatch>
+        </filter>
         <encoder>
             <pattern>${pattern}</pattern>
         </encoder>


Test patch: