Repository: SCons/scons

Issue:
CConditionalScanner does not detect included header from a command line macro
This issue is generated as a result of investigating using the `CConditionalScanner` to detect dependencies when the `#include` uses a macro, after a Discord discussion here: https://discord.com/channels/571796279483564041/571796280146133047/998690474216927345. This base problem has actually been recorded in several past issues, including #344 #2601 #3038.  

The investigation shows that if the macro is defined in the source file, like this:
```py
#define HEADER "foo.h"
#include HEADER
```
then `CConditionalScanner` indeed produces the dependency, but if it is defined as a compiler command-line option, it is not.
```py
env = Environment(CPPDEFINES=['HEADER=\\"foo.h\\"'])
```

Attaching a diff to the unit test file for the C scanner, which demonstrates this behavior (unfortunately had to make into a file type github would accept)

[CTests-diff.zip](https://github.com/SCons/scons/files/9143148/CTests-diff.zip)





Discussion:
Hi @mwichmann, do you use the latest development version? It should be fixed by https://github.com/SCons/scons/commit/d9192dbf0c4380043759262147e6e06b0caf590e
Doesn't seem to be... which doesn't mean my test isn't faulty.  I still get:
```
AssertionError: False is not true : expect ['f9.h'] != scanned []
```

I think this may have been fixed by #4263 - not an issue with the conditional scanner, but with the way `processDefines` emitted a single string-valued macro.  I'll need to swing by and check in more detail, but we may be able to close this.
Okay, reran the test case attached here, it still fails, but can be made to pass by:
```diff
-        env = DummyEnvironment(CPPDEFINES=['HEADER=\\"f9.h\\"'])
+        env = DummyEnvironment(CPPDEFINES=[('HEADER', '\\"f9.h\\"')])
```

So there may *still* be a `CPPDEFINES` hole here. 
The hole is in `SCons/Scanner/C/dictifyCPPDEFINES()` - it does its own processing of defines, not using the `processDefines` routine the rest of SCons uses, and did not deal with `"NAME=VALUE"` strings, either as a solo value, or as a member of a sequence. (note this problem is *not* related to the large `CPPDEFINES` refactor, it was always possible to assign a single string or a string-in-a-list to `CPPDEFINES`, only after type conversion is triggered might you not have those anymore).  I have a patch and some additional tests.

Resolving patch:
diff --git a/SCons/Scanner/C.py b/SCons/Scanner/C.py
--- a/SCons/Scanner/C.py
+++ b/SCons/Scanner/C.py
@@ -21,7 +21,12 @@
 # OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
-"""Dependency scanner for C/C++ code."""
+"""Dependency scanner for C/C++ code.
+
+Two scanners are defined here: the default CScanner, and the optional
+CConditionalScanner, which must be explicitly selected by calling
+add_scanner() for each affected suffix.
+"""
 
 import SCons.Node.FS
 import SCons.cpp
@@ -62,25 +67,60 @@ def read_file(self, file):
             return ''
 
 def dictify_CPPDEFINES(env) -> dict:
-    """Returns CPPDEFINES converted to a dict."""
+    """Returns CPPDEFINES converted to a dict.
+
+    This should be similar to :func:`~SCons.Defaults.processDefines`.
+    Unfortunately, we can't do the simple thing of calling that routine and
+    passing the result to the dict() constructor, because it turns the defines
+    into a list of "name=value" pairs, which the dict constructor won't
+    consume correctly.  Also cannot just call dict on CPPDEFINES itself - it's
+    fine if it's stored in the converted form (currently deque of tuples), but
+    CPPDEFINES could be in other formats too.
+
+    So we have to do all the work here - keep concepts in sync with
+    ``processDefines``.
+    """
     cppdefines = env.get('CPPDEFINES', {})
+    result = {}
     if cppdefines is None:
-        return {}
+        return result
+
+    if SCons.Util.is_Tuple(cppdefines):
+        try:
+            return {cppdefines[0]: cppdefines[1]}
+        except IndexError:
+            return {cppdefines[0]: None}
+
     if SCons.Util.is_Sequence(cppdefines):
-        result = {}
         for c in cppdefines:
             if SCons.Util.is_Sequence(c):
                 try:
                     result[c[0]] = c[1]
                 except IndexError:
-                    # it could be a one-item sequence
+                    # could be a one-item sequence
                     result[c[0]] = None
+            elif SCons.Util.is_String(c):
+                try:
+                    name, value = c.split('=')
+                    result[name] = value
+                except ValueError:
+                    result[c] = None
             else:
+                # don't really know what to do here
                 result[c] = None
         return result
-    if not SCons.Util.is_Dict(cppdefines):
-        return {cppdefines : None}
-    return cppdefines
+
+    if SCons.Util.is_String(cppdefines):
+        try:
+            name, value = cppdefines.split('=')
+            return {name: value}
+        except ValueError:
+            return {cppdefines: None}
+
+    if SCons.Util.is_Dict(cppdefines):
+        return cppdefines
+
+    return {cppdefines: None}
 
 class SConsCPPScannerWrapper:
     """The SCons wrapper around a cpp.py scanner.
diff --git a/SCons/Scanner/CTests.py b/SCons/Scanner/CTests.py
--- a/SCons/Scanner/CTests.py
+++ b/SCons/Scanner/CTests.py
@@ -91,6 +91,27 @@
 }
 """)
 
+# include using a macro, defined in source file
+test.write('f9a.c', """\
+#define HEADER "f9.h"
+#include HEADER
+
+int main(void)
+{
+   return 0;
+}
+""")
+
+# include using a macro, not defined in source file
+test.write('f9b.c', """\
+#include HEADER
+
+int main(void)
+{
+   return 0;
+}
+""")
+
 
 # for Emacs -> "
 
@@ -99,7 +120,7 @@
 headers = ['f1.h','f2.h', 'f3-test.h', 'fi.h', 'fj.h', 'never.h',
            'd1/f1.h', 'd1/f2.h', 'd1/f3-test.h', 'd1/fi.h', 'd1/fj.h',
            'd1/d2/f1.h', 'd1/d2/f2.h', 'd1/d2/f3-test.h',
-           'd1/d2/f4.h', 'd1/d2/fi.h', 'd1/d2/fj.h']
+           'd1/d2/f4.h', 'd1/d2/fi.h', 'd1/d2/fj.h', 'f9.h']
 
 for h in headers:
     test.write(h, " ")
@@ -224,7 +245,7 @@ def deps_match(self, deps, headers):
     global my_normpath
     scanned = list(map(my_normpath, list(map(str, deps))))
     expect = list(map(my_normpath, headers))
-    self.assertTrue(scanned == expect, "expect %s != scanned %s" % (expect, scanned))
+    self.assertTrue(scanned == expect, f"expect {expect} != scanned {scanned}")
 
 # define some tests:
 
@@ -443,7 +464,7 @@ def runTest(self):
         env = DummyEnvironment(CPPSUFFIXES = suffixes)
         s = SCons.Scanner.C.CScanner()
         for suffix in suffixes:
-            assert suffix in s.get_skeys(env), "%s not in skeys" % suffix
+            assert suffix in s.get_skeys(env), f"{suffix} not in skeys"
 
 
 class CConditionalScannerTestCase1(unittest.TestCase):
@@ -490,42 +511,70 @@ def runTest(self):
         headers = ['d1/f1.h']
         deps_match(self, deps, headers)
 
+
+class CConditionalScannerTestCase4(unittest.TestCase):
+    def runTest(self):
+        """Test that dependency is detected if #include uses a macro."""
+
+        with self.subTest("macro defined in the source file"):
+            env = DummyEnvironment()
+            s = SCons.Scanner.C.CConditionalScanner()
+            deps = s(env.File('f9a.c'), env, s.path(env))
+            headers = ['f9.h']
+            deps_match(self, deps, headers)
+
+        with self.subTest("macro defined on the command line"):
+            env = DummyEnvironment(CPPDEFINES='HEADER=\\"f9.h\\"')
+            #env = DummyEnvironment(CPPDEFINES=['HEADER=\\"f9.h\\"'])
+            s = SCons.Scanner.C.CConditionalScanner()
+            deps = s(env.File('f9b.c'), env, s.path(env))
+            headers = ['f9.h']
+            deps_match(self, deps, headers)
+
+
 class dictify_CPPDEFINESTestCase(unittest.TestCase):
     def runTest(self):
-        """Make sure single-item tuples convert correctly.
+        """Make sure CPPDEFINES converts correctly.
 
-        This is a regression test: AppendUnique turns sequences into
-        lists of tuples, and dictify could gack on these.
+        Various types and combinations of types could fail if not handled
+        specifically by dictify_CPPDEFINES - this is a regression test.
         """
-        env = DummyEnvironment(CPPDEFINES=(("VALUED_DEFINE", 1), ("UNVALUED_DEFINE", )))
-        d = SCons.Scanner.C.dictify_CPPDEFINES(env)
-        expect = {'VALUED_DEFINE': 1, 'UNVALUED_DEFINE': None}
-        assert d == expect
-
-def suite():
-    suite = unittest.TestSuite()
-    suite.addTest(CScannerTestCase1())
-    suite.addTest(CScannerTestCase2())
-    suite.addTest(CScannerTestCase3())
-    suite.addTest(CScannerTestCase4())
-    suite.addTest(CScannerTestCase5())
-    suite.addTest(CScannerTestCase6())
-    suite.addTest(CScannerTestCase8())
-    suite.addTest(CScannerTestCase9())
-    suite.addTest(CScannerTestCase10())
-    suite.addTest(CScannerTestCase11())
-    suite.addTest(CScannerTestCase12())
-    suite.addTest(CScannerTestCase13())
-    suite.addTest(CScannerTestCase14())
-    suite.addTest(CScannerTestCase15())
-    suite.addTest(CConditionalScannerTestCase1())
-    suite.addTest(CConditionalScannerTestCase2())
-    suite.addTest(CConditionalScannerTestCase3())
-    suite.addTest(dictify_CPPDEFINESTestCase())
-    return suite
+        with self.subTest("tuples in a sequence, including one without value"):
+            env = DummyEnvironment(CPPDEFINES=[("VALUED", 1), ("UNVALUED",)])
+            d = SCons.Scanner.C.dictify_CPPDEFINES(env)
+            expect = {"VALUED": 1, "UNVALUED": None}
+            self.assertEqual(d, expect)
+
+        with self.subTest("tuple-valued define by itself"):
+            env = DummyEnvironment(CPPDEFINES=("STRING", "VALUE"))
+            d = SCons.Scanner.C.dictify_CPPDEFINES(env)
+            expect = {"STRING": "VALUE"}
+            self.assertEqual(d, expect)
+
+        with self.subTest("string-valued define in a sequence"):
+            env = DummyEnvironment(CPPDEFINES=[("STRING=VALUE")])
+            d = SCons.Scanner.C.dictify_CPPDEFINES(env)
+            expect = {"STRING": "VALUE"}
+            self.assertEqual(d, expect)
+
+        with self.subTest("string-valued define by itself"):
+            env = DummyEnvironment(CPPDEFINES="STRING=VALUE")
+            d = SCons.Scanner.C.dictify_CPPDEFINES(env)
+            expect = {"STRING": "VALUE"}
+            self.assertEqual(d, expect)
+
+        from collections import deque
+        with self.subTest("compound CPPDEFINES in internal format"):
+            env = DummyEnvironment(
+                CPPDEFINES=deque([("STRING", "VALUE"), ("UNVALUED",)])
+            )
+            d = SCons.Scanner.C.dictify_CPPDEFINES(env)
+            expect = {"STRING": "VALUE", "UNVALUED": None}
+            self.assertEqual(d, expect)
+
 
 if __name__ == "__main__":
-    TestUnit.run(suite())
+    unittest.main()
 
 # Local Variables:
 # tab-width:4


Test patch: