Repository: SCons/scons

Issue:
Java scanner questionable handling of JAVACLASSPATH
The handling of `JAVACLASSPATH` in the Java scanner by the `_subst_libs` function seems problematic. `_subst_libs` is a direct copy of the code in `SCons/Scanner/Prog.py` (down to the comments), where the `LIBS` construction variable is modified, but `JAVACLASSPATH` is a different beast: it can be base names for class files, which is somewhat library-like, but it can also be pathnames to search for class files. The problem area is that whatever the form it's passed in, `JAVACLASSPATH` is split on whitespace, this cannot be avoided as even if a list is passed, as it recursively splits the contents.  As a result, no path with a space in it is going to work.

A potential solution is to not split if a list is passed, which would align with a lot of other SCons behavior. Not even entirely sure that splitting if it's a string is correct. For example, the unit test does something like this:

```py
    def runTest(self):
        env = DummyEnvironment(JAVASUFFIXES=['.java'],  JAVACLASSPATH=[test.workpath()])
        s = SCons.Scanner.Java.JavaScanner()
```
So it actually passes a list.  But what if you did this:
```py
        env = DummyEnvironment(JAVASUFFIXES=['.java'],  JAVACLASSPATH=test.workpath('My Java'))
```
That is, generate an absolute path whose final component has a space in it, and *not* pass that path in a list - that would fail as splitting on space there gives you a list containing an absolute path ending in `My`, and a relative path `Java`. 

Nevertheless, we could certainly decide that a string with spaces is split, but a string with spaces as a list element is not, and document that, so a way forward is at least clear.  Note that the current manpage wording is a bit murky - it could make clearer that SCons adds the search path separator before it is emitted on the command line; the user should not use that separator in setting `JAVACLASSPATH`.

Note: this was originally raised in a Discord message https://discord.com/channels/571796279483564041/571796280146133047/1030578995953152120


Discussion:
Additional comment as far as the manpage entry: it suggests that `JAVACLASSPATH` is always directories, but the docstring in the Java scanner code has a more expansive set of rules:
```py
    """Scan for files on the JAVACLASSPATH.

    The classpath can contain:
     - Explicit paths to JAR/Zip files
     - Wildcards (*)
     - Directories which contain classes in an unnamed package
     - Parent directories of the root package for classes in a named package

     Class path entries that are neither directories nor archives (.zip
     or JAR files) nor the asterisk (*) wildcard character are ignored.
    """
```
Should more of this be reflected in the manpage?  The manpage only talks about its use in being emitted on the `javac` command line, 

Resolving patch:
diff --git a/SCons/Scanner/Java.py b/SCons/Scanner/Java.py
--- a/SCons/Scanner/Java.py
+++ b/SCons/Scanner/Java.py
@@ -21,56 +21,64 @@
 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
-import os 
+import os
 
 import SCons.Node
 import SCons.Node.FS
 import SCons.Scanner
-import SCons.Util
+from SCons.Util import flatten, is_String
 
 
-def _subst_libs(env, libs):
-    """
-    Substitute environment variables and split into list.
+def _subst_paths(env, paths) -> list:
+    """Return a list of substituted path elements.
+
+    If *paths* is a string, it is split on the search-path separator.
+    Otherwise, substitution is done on string-valued list elements but
+    they are not split.
+
+    Note helps support behavior like pulling in the external ``CLASSPATH``
+    and setting it directly into ``JAVACLASSPATH``, however splitting on
+    ``os.pathsep`` makes the interpretation system-specific (this is
+    warned about in the manpage entry for ``JAVACLASSPATH``).
     """
-    if SCons.Util.is_String(libs):
-        libs = env.subst(libs)
-        if SCons.Util.is_String(libs):
-            libs = libs.split()
-    elif SCons.Util.is_Sequence(libs):
-        _libs = []
-        for lib in libs:
-            _libs += _subst_libs(env, lib)
-        libs = _libs
+    if is_String(paths):
+        paths = env.subst(paths)
+        if SCons.Util.is_String(paths):
+            paths = paths.split(os.pathsep)
     else:
-        # libs is an object (Node, for example)
-        libs = [libs]
-    return libs
+        # TODO: may want to revisit splitting list-element strings if requested
+        paths = flatten(paths)
+        paths = [env.subst(path) if is_String(path) else path for path in paths]
+    return paths
 
 
-def _collect_classes(list, dirname, files):
+def _collect_classes(classlist, dirname, files):
     for fname in files:
-        if os.path.splitext(fname)[1] == ".class":
-            list.append(os.path.join(str(dirname), fname))
+        if fname.endswith(".class"):
+            classlist.append(os.path.join(str(dirname), fname))
 
 
-def scan(node, env, libpath=()):
+def scan(node, env, libpath=()) -> list:
     """Scan for files on the JAVACLASSPATH.
 
-    The classpath can contain:
+    JAVACLASSPATH path can contain:
      - Explicit paths to JAR/Zip files
      - Wildcards (*)
      - Directories which contain classes in an unnamed package
      - Parent directories of the root package for classes in a named package
 
-     Class path entries that are neither directories nor archives (.zip or JAR files) nor the asterisk (*) wildcard character are ignored.
-     """
+    Class path entries that are neither directories nor archives (.zip
+    or JAR files) nor the asterisk (*) wildcard character are ignored.
+    """
     classpath = env.get('JAVACLASSPATH', [])
-    classpath = _subst_libs(env, classpath)
+    classpath = _subst_paths(env, classpath)
 
     result = []
     for path in classpath:
-        if SCons.Util.is_String(path) and "*" in path:
+        if is_String(path) and "*" in path:
+            # This matches more than the Java docs describe: a '*' only
+            # matches jar files. The filter later should trim this down.
+            # TODO: should we filter here? use .endswith('*') rather than "in"?
             libs = env.Glob(path)
         else:
             libs = [path]
@@ -89,8 +97,11 @@ def scan(node, env, libpath=()):
 
 
 def JavaScanner():
-    return SCons.Scanner.Base(scan, 'JavaScanner',
-                              skeys=['.java'])
+    """Scanner for .java files.
+
+    .. versionadded:: 4.4
+    """
+    return SCons.Scanner.Base(scan, 'JavaScanner', skeys=['.java'])
 
 # Local Variables:
 # tab-width:4
diff --git a/SCons/Scanner/JavaTests.py b/SCons/Scanner/JavaTests.py
--- a/SCons/Scanner/JavaTests.py
+++ b/SCons/Scanner/JavaTests.py
@@ -33,18 +33,25 @@
 
 
 test = TestCmd.TestCmd(workdir = '')
-test.subdir('com')
 
 files = [
     'bootclasspath.jar',
     'classpath.jar',
     'Test.class',
-    'com/Test.class'
 ]
 
 for fname in files:
     test.write(fname, "\n")
 
+test.subdir('com')
+test.subdir('java space')
+subfiles = [
+    'com/Test.class',
+    'java space/Test.class'
+]
+
+for fname in subfiles:
+    test.write(fname.split('/'), "\n")
 
 class DummyEnvironment(collections.UserDict):
     def __init__(self,**kw):
@@ -52,7 +59,7 @@ def __init__(self,**kw):
         self.data.update(kw)
         self.fs = SCons.Node.FS.FS(test.workpath(''))
         self['ENV'] = {}
-        
+
     def Dictionary(self, *args):
         return self.data
 
@@ -80,7 +87,7 @@ def Dir(self, filename):
 
     def File(self, filename):
         return self.fs.File(filename)
-    
+
     def Glob(self, path):
         return self.fs.Glob(path)
 
@@ -111,8 +118,7 @@ def deps_match(self, deps, headers):
 class JavaScannerEmptyClasspath(unittest.TestCase):
     def runTest(self):
         path = []
-        env = DummyEnvironment(JAVASUFFIXES=['.java'],
-                               JAVACLASSPATH=path)
+        env = DummyEnvironment(JAVASUFFIXES=['.java'], JAVACLASSPATH=path)
         s = SCons.Scanner.Java.JavaScanner()
         deps = s(DummyNode('dummy'), env)
         expected = []
@@ -145,10 +151,33 @@ def runTest(self):
                                JAVACLASSPATH=[test.workpath()])
         s = SCons.Scanner.Java.JavaScanner()
         deps = s(DummyNode('dummy'), env)
-        expected = ['Test.class', 'com/Test.class']
+        expected = ['Test.class', 'com/Test.class', 'java space/Test.class']
+        deps_match(self, deps, expected)
+
+
+class JavaScannerNamedDirClasspath(unittest.TestCase):
+    def runTest(self):
+        env = DummyEnvironment(
+            JAVASUFFIXES=['.java'],
+            JAVACLASSPATH=[test.workpath('com'), test.workpath('java space')],
+        )
+        s = SCons.Scanner.Java.JavaScanner()
+        deps = s(DummyNode('dummy'), env)
+        expected = ['com/Test.class', 'java space/Test.class']
         deps_match(self, deps, expected)
 
 
+class JavaScannerSearchPathClasspath(unittest.TestCase):
+    def runTest(self):
+        env = DummyEnvironment(
+            JAVASUFFIXES=['.java'],
+            JAVACLASSPATH=os.pathsep.join([test.workpath('com'), test.workpath('java space')]),
+        )
+        s = SCons.Scanner.Java.JavaScanner()
+        deps = s(DummyNode('dummy'), env)
+        expected = ['com/Test.class', 'java space/Test.class']
+        deps_match(self, deps, expected)
+
 
 if __name__ == "__main__":
     unittest.main()


Test patch: