Repository: SCons/scons

Issue:
CompilationDatabase tool emits paths into variant directories for source files when not using absolute paths.
**Describe the bug**
When using the CompilationDatabase tool with `COMPILATIONDB_USE_ABSPATH` == False, the `file` field appears to refer into the variant directory, not the source directory:

`"file": "build/cached/third_party/s2/base/int128.cc",`

Instead of 

`"file": "src/third_party/s2/base/int128.cc"`

**Required information**
* Link to SCons Users thread discussing your issue.
Discussed directly with @bdbaddog 

* Version of SCons
master

* Version of Python
3.7

* How you installed SCons
source

* What Platform are you on? (Linux/Windows and which version)
macOS, but shouldn't matter.



Discussion:
@acmorrow  - So you'd expect the source file to point to source of variantdir, but target to point to variant dir itself?
@bdbaddog - I'm not following you there. The error is in the `file` field of the JSON result. It is referring to a source file by a path into the variant directory. That would only be true if you were building with `duplicate=1` or similar. The `file` field should have the path to the source file where it lives on disk. I'm not sure what you mean by "target to point to variant dir itself".
@acmorrow like this
```
% cat compile_commands.json
[
    {
        "command": "/Users/bdbaddog/.virtualenvs/scons37/bin/python mygcc.py cc -o build/test_main.o -c build/test_main.c",
        "directory": "/private/var/folders/ng/48pttrpj239fw5rmm3x65pxr0000gn/T/testcmd.32413.myyo97s5",
        "file": "src/test_main.c",
        "output": "build/test_main.o"
    }
]%
```
Yes, that looks like what I would expect for `command`, `directory`, and `file`. If you provide a fix that adjusts that, I'll re-run my experiment to A/B compare against the MongoDB one and see that the output is now aligned.

Regarding the `output` field: I wonder. I'm not sure if it is actually supposed to be a path or not. The "spec" doesn't really say: it says it should be "The name of the output created by this compilation step". Does that include the path? Who knows?
I'm sure once we release it we'll get feedback.. ;)
Adding a test for variant_dir with duplicate = True and False..

Resolving patch:
diff --git a/SCons/Tool/compilation_db.py b/SCons/Tool/compilation_db.py
--- a/SCons/Tool/compilation_db.py
+++ b/SCons/Tool/compilation_db.py
@@ -142,10 +142,10 @@ def write_compilation_db(target, source, env):
         output_file = entry['output']
 
         if use_abspath:
-            source_file = source_file.abspath
+            source_file = source_file.srcnode().abspath
             output_file = output_file.abspath
         else:
-            source_file = source_file.path
+            source_file = source_file.srcnode().path
             output_file = output_file.path
 
         path_entry = {'directory': entry['directory'],


Test patch:
diff --git a/test/CompilationDatabase/fixture/SConstruct_variant b/test/CompilationDatabase/fixture/SConstruct_variant
new file mode 100644
--- /dev/null
+++ b/test/CompilationDatabase/fixture/SConstruct_variant
@@ -0,0 +1,40 @@
+import sys
+
+DefaultEnvironment(tools=[])
+env = Environment(
+    PYTHON=sys.executable,
+    LINK='$PYTHON mylink.py',
+    LINKFLAGS=[],
+    CC='$PYTHON mygcc.py cc',
+    CXX='$PYTHON mygcc.py c++',
+    tools=['gcc','g++','gnulink'],
+)
+env.Tool('compilation_db')
+
+env_abs = env.Clone(COMPILATIONDB_USE_ABSPATH=True)
+env_abs.CompilationDatabase('compile_commands_clone_abs.json')
+
+# Should be relative paths
+env.CompilationDatabase('compile_commands_only_arg.json')
+env.CompilationDatabase(target='compile_commands_target.json')
+
+# Should default name compile_commands.json
+env.CompilationDatabase()
+
+# Should be absolute paths
+env.CompilationDatabase('compile_commands_over_abs.json', COMPILATIONDB_USE_ABSPATH=True)
+env.CompilationDatabase(target='compile_commands_target_over_abs.json', COMPILATIONDB_USE_ABSPATH=True)
+
+# Should be relative paths
+env.CompilationDatabase('compile_commands_over_rel.json', COMPILATIONDB_USE_ABSPATH=False)
+
+# Try 1/0 for COMPILATIONDB_USE_ABSPATH
+env.CompilationDatabase('compile_commands_over_abs_1.json', COMPILATIONDB_USE_ABSPATH=1)
+env.CompilationDatabase('compile_commands_over_abs_0.json', COMPILATIONDB_USE_ABSPATH=0)
+
+env.VariantDir('build','src')
+env.Program('build/main', 'build/test_main.c')
+
+env.VariantDir('build2','src', duplicate=0)
+env.Program('build2/main', 'build2/test_main.c')
+
diff --git a/test/CompilationDatabase/variant_dir.py b/test/CompilationDatabase/variant_dir.py
new file mode 100644
--- /dev/null
+++ b/test/CompilationDatabase/variant_dir.py
@@ -0,0 +1,118 @@
+#!/usr/bin/env python
+#
+# __COPYRIGHT__
+#
+# Permission is hereby granted, free of charge, to any person obtaining
+# a copy of this software and associated documentation files (the
+# "Software"), to deal in the Software without restriction, including
+# without limitation the rights to use, copy, modify, merge, publish,
+# distribute, sublicense, and/or sell copies of the Software, and to
+# permit persons to whom the Software is furnished to do so, subject to
+# the following conditions:
+#
+# The above copyright notice and this permission notice shall be included
+# in all copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
+# KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
+# WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+#
+"""
+Test CompilationDatabase and several variations of ways to call it
+and values of COMPILATIONDB_USE_ABSPATH
+"""
+
+import sys
+import os
+import os.path
+import TestSCons
+
+test = TestSCons.TestSCons()
+
+test.file_fixture('mylink.py')
+test.file_fixture('mygcc.py')
+
+test.verbose_set(1)
+test.file_fixture('fixture/SConstruct_variant', 'SConstruct')
+test.file_fixture('test_main.c', 'src/test_main.c')
+test.run()
+
+rel_files = [
+    'compile_commands_only_arg.json',
+    'compile_commands_target.json',
+    'compile_commands.json',
+    'compile_commands_over_rel.json',
+    'compile_commands_over_abs_0.json'
+]
+
+abs_files = [
+    'compile_commands_clone_abs.json',
+    'compile_commands_over_abs.json',
+    'compile_commands_target_over_abs.json',
+    'compile_commands_over_abs_1.json',
+]
+
+example_rel_file = """[
+    {
+        "command": "%(exe)s mygcc.py cc -o %(output_file)s -c %(variant_src_file)s",
+        "directory": "%(workdir)s",
+        "file": "%(src_file)s",
+        "output": "%(output_file)s"
+    },
+    {
+        "command": "%(exe)s mygcc.py cc -o %(output2_file)s -c %(src_file)s",
+        "directory": "%(workdir)s",
+        "file": "%(src_file)s",
+        "output": "%(output2_file)s"
+    }
+]""" % {'exe': sys.executable,
+        'workdir': test.workdir,
+        'src_file': os.path.join('src', 'test_main.c'),
+        'output_file': os.path.join('build', 'test_main.o'),
+        'output2_file': os.path.join('build2', 'test_main.o'),
+        'variant_src_file': os.path.join('build', 'test_main.c')
+        }
+
+if sys.platform == 'win32':
+    example_rel_file = example_rel_file.replace('\\', '\\\\')
+
+for f in rel_files:
+    # print("Checking:%s" % f)
+    test.must_exist(f)
+    test.must_match(f, example_rel_file, mode='r')
+
+example_abs_file = """[
+    {
+        "command": "%(exe)s mygcc.py cc -o %(output_file)s -c %(variant_src_file)s",
+        "directory": "%(workdir)s",
+        "file": "%(abs_src_file)s",
+        "output": "%(abs_output_file)s"
+    },
+    {
+        "command": "%(exe)s mygcc.py cc -o %(output2_file)s -c %(src_file)s",
+        "directory": "%(workdir)s",
+        "file": "%(abs_src_file)s",
+        "output": "%(abs_output2_file)s"
+    }
+]""" % {'exe': sys.executable,
+        'workdir': test.workdir,
+        'src_file': os.path.join('src', 'test_main.c'),
+        'abs_src_file': os.path.join(test.workdir, 'src', 'test_main.c'),
+        'abs_output_file': os.path.join(test.workdir, 'build', 'test_main.o'),
+        'abs_output2_file': os.path.join(test.workdir, 'build2', 'test_main.o'),
+        'output_file': os.path.join('build', 'test_main.o'),
+        'output2_file': os.path.join('build2', 'test_main.o'),
+        'variant_src_file': os.path.join('build', 'test_main.c')}
+
+if sys.platform == 'win32':
+    example_abs_file = example_abs_file.replace('\\', '\\\\')
+
+for f in abs_files:
+    test.must_exist(f)
+    test.must_match(f, example_abs_file, mode='r')
+
+test.pass_test()