Repository: OpenRefine/OpenRefine (java)

Issue:
Add 'refine(.bat) lint' command
Now that we are enforcing some formatting settings (for now only in Java) it would be good to offer a simple command that contributors could use to lint their changes before submitting them as a PR. That's useful when making changes from an editor that does not support (or is not appropriately configured to follow) our enforced style.

### Proposed solution

Introduce a new `lint` command in `./refine` and `refine.bat` to lint all files according to our required format.
For now it would amount to running `mvn formatter:format impsort:sort`.

### Alternatives considered

We could attempt to do that automatically instead:
* as a git filter, registered in `.gitattributes`. This would be optimal in terms of developer experience, but is only workable if we are able to offer an executable which quickly lints a file (provided on standard input, output on standard output). The Maven plugins are not designed for this use case. I have been experimenting with Graal's Native image functionality to see if this would help, I might make a blog post about that soon
* as a git pre-commit hook. This is less convenient, but in such a workflow it is a bit less annoying that the linting process takes a few seconds. I have been experimenting with this, but the hook I have been developing is still not behaving like it should in some cases, so I don't think it's something I would encourage others to use. Also I think the git filter approach is more fitting
* as an operation that's included in other goals. For instance we could bind the `formatter:format` and `impsort:sort` goals to some phases of the Maven lifecycle, so that they are executed automatically when compiling or testing.
* as a GitHub bot which automatically adds a linting commit to PRs which are not readily linted

### Additional context
Follow-up to #6108. 


Pull request:
Add 'lint' option to refine(.bat) scripts
This helps developers reformat their code according to our standards.

Closes #6166.

Not tested on Windows yet.


Patch:
diff --git a/extensions/gdata/pom.xml b/extensions/gdata/pom.xml
index 59375016a..2ed919c1b 100644
--- a/extensions/gdata/pom.xml
+++ b/extensions/gdata/pom.xml
@@ -84,7 +84,7 @@
     <dependency>
       <groupId>com.google.apis</groupId>
       <artifactId>google-api-services-drive</artifactId>
-      <version>v3-rev20231120-2.0.0</version>
+      <version>v3-rev20231128-2.0.0</version>
     </dependency>
     <dependency>
       <groupId>com.google.apis</groupId>
diff --git a/extensions/wikibase/src/org/openrefine/wikibase/qa/scrutinizers/FileNameScrutinizer.java b/extensions/wikibase/src/org/openrefine/wikibase/qa/scrutinizers/FileNameScrutinizer.java
index 353d3a3f5..a54c8eaf8 100644
--- a/extensions/wikibase/src/org/openrefine/wikibase/qa/scrutinizers/FileNameScrutinizer.java
+++ b/extensions/wikibase/src/org/openrefine/wikibase/qa/scrutinizers/FileNameScrutinizer.java
@@ -22,7 +22,7 @@ public class FileNameScrutinizer extends EditScrutinizer {
     // see https://commons.wikimedia.org/wiki/Commons:File_naming
     public static final int maxFileNameBytes = 240;
     public static final Pattern forbiddenFileNameChars = Pattern.compile(
-            ".*([^ %!\"$&'()*,\\-./\\d:;=?@\\p{L}\\p{Mc}\\\\^_`~\\x80-\\xFF+]|%[0-9A-Fa-f]{2}|&[A-Za-z0-9\\x80-\\xff]+;|&#[0-9]+;|&#x[0-9A-Fa-f]+;).*");
+            ".*([^ %!\"$&'()*,\\-./\\d:;=?@\\p{L}\\p{M}\\p{N}\\\\^_`~\\x80-\\xFF+]|%[0-9A-Fa-f]{2}|&[A-Za-z0-9\\x80-\\xff]+;|&#[0-9]+;|&#x[0-9A-Fa-f]+;).*");
 
     public static final String duplicateFileNamesInBatchType = "duplicate-file-names-in-batch";
     public static final String fileNamesAlreadyExistOnWikiType = "file-names-already-exist-on-wiki";
@@ -98,7 +98,7 @@ public class FileNameScrutinizer extends EditScrutinizer {
             // Invalid characters
             Matcher matcher = forbiddenFileNameChars.matcher(fileName);
             if (matcher.matches()) {
-                QAWarning issue = new QAWarning(invalidCharactersInFileNameType, null, QAWarning.Severity.CRITICAL,
+                QAWarning issue = new QAWarning(invalidCharactersInFileNameType, null, QAWarning.Severity.IMPORTANT,
                         1);
                 issue.setProperty("example_filename", fileName);
                 issue.setProperty("invalid_character", matcher.group(1));
diff --git a/refine b/refine
index 58cdda042..08c9208d4 100755
--- a/refine
+++ b/refine
@@ -42,17 +42,18 @@ Options
     --jmx                         Enable JMX monitoring.
 
 Actions
+    run                           Run OpenRefine [default].
     build                         Build OpenRefine.
     clean                         Clean compiled classes.
-    dist <version>                Make all distributions.
+    test                          Run all tests.
     extensions_test               Run the extensions tests.
-    linux_dist <version>          Make Linux binary distribution.
-    mac_dist <version>            Make MacOSX binary distribution.
-    run                           Run OpenRefine [default].
     server_test                   Run the server tests.
-    test                          Run all tests.
-    e2e_tests                      Run the e2e tests.
+    e2e_tests                     Run the e2e tests.
+    lint                          Reformat the source code according to OpenRefine's conventions.
+    mac_dist <version>            Make MacOS binary distribution.
     windows_dist <version>        Make Windows binary distribution.
+    linux_dist <version>          Make Linux binary distribution.
+    dist <version>                Make all distributions.
 EOF
 exit 1
 }
@@ -244,6 +245,13 @@ do_mvn() {
 
 # ----------------------------------------------------------------------------------------------
 
+lint() {
+    mvn_prepare
+    "$MVN" formatter:format impsort:sort
+}
+
+# ----------------------------------------------------------------------------------------------
+
 dist() {
     get_version $1
     build_prepare
@@ -720,12 +728,13 @@ case "$ACTION" in
   clean) do_mvn clean;;
   test) test $1;;
   tests) test $1;;
-  e2e_tests) e2e_tests;;    
+  e2e_tests) e2e_tests;;
   server_test) server_test $1;;  
   server_tests) server_test $1;;  
   extensions_test) extensions_test $1;;
   extensions_tests) extensions_test $1;;
   run) run;;
+  lint) lint;;
   mac_dist) mac_dist $1;;
   windows_dist) windows_dist $1;;
   linux_dist) linux_dist $1;;
diff --git a/refine.bat b/refine.bat
index ef9361f6f..3e6768dfd 100644
--- a/refine.bat
+++ b/refine.bat
@@ -48,6 +48,7 @@ echo     extensions_test         Run the extensions tests.
 echo     run                     Run OpenRefine.
 echo     server_test             Run the server tests.
 echo     test                    Run all the tests.
+echo     lint                    Reformat the source code according to OpenRefine's conventions.
 goto :eof
 
 
@@ -190,6 +191,7 @@ if ""%ACTION%"" == ""build"" goto doMvn
 if ""%ACTION%"" == ""server_test"" goto doMvn
 if ""%ACTION%"" == ""extensions_test"" goto doMvn
 if ""%ACTION%"" == ""test"" goto doMvn
+if ""%ACTION%"" == ""lint"" goto doMvn
 if ""%ACTION%"" == ""clean"" goto doMvn
 if ""%ACTION%"" == ""run"" goto doRun
 if ""%ACTION%"" == """" goto doRun
@@ -270,6 +272,12 @@ set MVN_ACTION=compile test-compile dependency:build-classpath
 if ""%ACTION%"" == ""test"" set MVN_ACTION=test dependency:build-classpath
 if ""%ACTION%"" == ""server_test"" set MVN_ACTION=test -f main
 if ""%ACTION%"" == ""extensions_test"" set MVN_ACTION=test -f extensions
+if ""%ACTION%"" == ""lint"" (
+    set MVN_ACTION=formatter:format impsort:sort
+    rem Skip the call to process-resources as it's not needed for this action
+    goto :mvnCall
+)
 call "%MVN%" process-resources
+:mvnCall
 call "%MVN%" %MVN_ACTION%
 goto :eof


Test patch:
diff --git a/extensions/wikibase/tests/src/org/openrefine/wikibase/qa/scrutinizers/FileNameScrutinizerTest.java b/extensions/wikibase/tests/src/org/openrefine/wikibase/qa/scrutinizers/FileNameScrutinizerTest.java
index abb1da6c4..5ced94b25 100644
--- a/extensions/wikibase/tests/src/org/openrefine/wikibase/qa/scrutinizers/FileNameScrutinizerTest.java
+++ b/extensions/wikibase/tests/src/org/openrefine/wikibase/qa/scrutinizers/FileNameScrutinizerTest.java
@@ -78,6 +78,16 @@ public class FileNameScrutinizerTest extends ScrutinizerTest {
         assertNoWarningRaised();
     }
 
+    @Test
+    public void testValidCharactersInFilenameNonAscii() {
+        MediaInfoEdit edit = new MediaInfoEditBuilder(TestingData.newMidA)
+                .addFileName("সমাচার দর্পণ - ৮ অক্টোবর ১৮৩৬.pdf")
+                .build();
+
+        scrutinize(edit);
+        assertNoWarningRaised();
+    }
+
     @Test()
     public void testInvalidCharactersInFilenameTab() throws IOException, MediaWikiApiErrorException {
         MediaInfoEdit edit = new MediaInfoEditBuilder(TestingData.newMidA)