Repository: rust
Issue: DWARF: linkage_name does not include hash, does not match any symbol
Labels: A-linkage,A-debuginfo,T-compiler,C-bug

So this is going to be a long issue, but the gist is, to put it semi dramatically, is that I think that all of rust debugging info might be slightly broken, but workable enough for say gdb, that it has gone unnoticed. At the very least, I think:

1. no_mangle statics are definitely broken https://github.com/rust-lang/rust/issues/33172
2. there are some major discrepancies between rust and c++ dwarf output that should be resolved, w.r.t. the DWARF `linkage_name`

/cc @philipc @fitzgen @tromey @rkruppe @michaelwoerister

# Discussion

I've created some test files in rust and c++, and also grepped the binaries for dwarf dies and also symbol table values, which I explain below.

## Repro / Test files

```rust
#[derive(Debug)]
pub struct Foo {
    x: u64,
    y: i32,
}

#[no_mangle]
pub static TEST: Foo = Foo { x: 0xdeadbeef, y: -55 };

pub static TEST2: Foo = Foo { x: 0xbeefdead, y: -55 };

fn deadbeef() {
    println!("TEST: {:?} - {:?}", TEST, TEST2);
}

pub fn main() {
    deadbeef()
}
```

and an approximating C++ file:

```c++
#include<cstdio>
#include<cstdint>

namespace test {

  struct Foo {
    uint64_t x;
    int64_t y;
  };

  Foo TEST = { .x = 0xdeadbeef, .y = -55 };
}

test::Foo TEST = { .x = 0xbeefdead, .y = -55 };

namespace test {
  void deadbeef() {
    printf("test::TEST: {0x%lx, %ld}\n", test::TEST.x, test::TEST.y);
    printf("TEST: {0x%lx, %ld}\n", TEST.x, TEST.y);
  }
}

int main() {
  test::deadbeef();
  return 0;
}
```

I've compiled the rust and c++ versions as follows:

```
rustc -g test.rs -o test
g++ -g -std=c++11 test.cpp -o test_cpp
clang++ -g -std=c++11 test.cpp -o test_cpp_clang
```

I will use the clang output for the c++ examples below, since it shares the same backend infrastructure, though the g++ does output the same.

## Analysis

First I will show the dwarf values for `TEST`, `TEST2`, and the function `deadbeef` for the `test` binary, then the `test_cpp_clang` binary.

I would like to direct the reader's attention to the `DW_AT_linkage_name` field, and the corresponding linkage name it shows for each binary.

```
DW_TAG_variable [3]  
  DW_AT_name [DW_FORM_strp]	( .debug_str[0x00000061] = "TEST")
  DW_AT_type [DW_FORM_ref4]	(cu + 0x0049 => {0x00000049})
  DW_AT_external [DW_FORM_flag_present]	(true)
  DW_AT_decl_file [DW_FORM_data1]	("/home/m4b/tmp/bad_debug/test.rs")
  DW_AT_decl_line [DW_FORM_data1]	(8)
  DW_AT_alignment [DW_FORM_udata]	(1)
  DW_AT_location [DW_FORM_exprloc]	(<0x9> 03 c0 65 05 00 00 00 00 00 )
  DW_AT_linkage_name [DW_FORM_strp]	( .debug_str[0x00000066] = "_ZN4test4TESTE")

DW_TAG_variable [6]  
  DW_AT_name [DW_FORM_strp]	( .debug_str[0x00000075] = "TEST2")
  DW_AT_type [DW_FORM_ref4]	(cu + 0x0049 => {0x00000049})
  DW_AT_decl_file [DW_FORM_data1]	("/home/m4b/tmp/bad_debug/test.rs")
  DW_AT_decl_line [DW_FORM_data1]	(10)
  DW_AT_alignment [DW_FORM_udata]	(1)
  DW_AT_location [DW_FORM_exprloc]	(<0x9> 03 d0 65 05 00 00 00 00 00 )
  DW_AT_linkage_name [DW_FORM_strp]	( .debug_str[0x0000007b] = "_ZN4test5TEST2E")

DW_TAG_subprogram [7] *
  DW_AT_low_pc [DW_FORM_addr]	(0x00000000000073b0)
  DW_AT_high_pc [DW_FORM_data4]	(0x000000f7)
  DW_AT_frame_base [DW_FORM_exprloc]	(<0x1> 56 )
  DW_AT_linkage_name [DW_FORM_strp]	( .debug_str[0x000000d9] = "_ZN4test8deadbeefE")
  DW_AT_name [DW_FORM_strp]	( .debug_str[0x000000ec] = "deadbeef")
  DW_AT_decl_file [DW_FORM_data1]	("/home/m4b/tmp/bad_debug/test.rs")
  DW_AT_decl_line [DW_FORM_data1]	(12)
```

And now for the cpp version:

```
DW_TAG_variable [3]  
  DW_AT_name [DW_FORM_strp]     ( .debug_str[0x00000053] = "TEST")
  DW_AT_type [DW_FORM_ref4]     (cu + 0x0048 => {0x00000048})
  DW_AT_external [DW_FORM_flag_present] (true)
  DW_AT_decl_file [DW_FORM_data1]       ("/home/m4b/tmp/bad_debug/test.cpp")
  DW_AT_decl_line [DW_FORM_data1]       (11)
  DW_AT_location [DW_FORM_exprloc]      (<0x9> 03 30 10 20 00 00 00 00 00 )
  DW_AT_linkage_name [DW_FORM_strp]     ( .debug_str[0x0000008e] = "_ZN4test4TESTE")

DW_TAG_subprogram [6]  
  DW_AT_low_pc [DW_FORM_addr]   (0x0000000000000670)
  DW_AT_high_pc [DW_FORM_data4] (0x0000004c)
  DW_AT_frame_base [DW_FORM_exprloc]    (<0x1> 56 )
  DW_AT_linkage_name [DW_FORM_strp]     ( .debug_str[0x000002be] = "_ZN4test8deadbeefEv")
  DW_AT_name [DW_FORM_strp]     ( .debug_str[0x000002d2] = "deadbeef")
  DW_AT_decl_file [DW_FORM_data1]       ("/home/m4b/tmp/bad_debug/test.cpp")
  DW_AT_decl_line [DW_FORM_data1]       (17)
  DW_AT_external [DW_FORM_flag_present] (true)

DW_TAG_variable [9]  
  DW_AT_name [DW_FORM_strp]       ( .debug_str[0x00000053] = "TEST")
  DW_AT_type [DW_FORM_ref4]       (cu + 0x0048 => {0x00000048})
  DW_AT_external [DW_FORM_flag_present]   (true)
  DW_AT_decl_file [DW_FORM_data1] ("/home/m4b/tmp/bad_debug/test.cpp")
  DW_AT_decl_line [DW_FORM_data1] (14)
  DW_AT_location [DW_FORM_exprloc]        (<0x9> 03 40 10 20 00 00 00 00 00 )
```

The first thing to note is that for the rust, `no_mangle` static, `TEST`, it is given a linkage name:

```
DW_AT_linkage_name [DW_FORM_strp]	( .debug_str[0x00000066] = "_ZN4test4TESTE")
```

In contrast to the cpp version, which (correctly) has none. I believe the rust DIE that is emitted is outright incorrect, and is the cause of the issue in https://github.com/rust-lang/rust/issues/33172

Note, although this issue was closed in favor of https://github.com/rust-lang/rust/issues/32574 that issue _does not_ `no_mangle` the static.

Unfortunately, that issue also noted (but did not seem to pursue further):

> Now, this variable is not actually emitted. There's no ELF symbol for it.

which I believe may be the crux of the major problem at large here: the `linkage_name` on **all non-mangled Rust DWARF DIEs** references a **non-existent** symbol - I think this is at best highly unusual, and at worst problematic.

## Missing Symbols

Considering only ELF at the moment, we can verify that for `TEST`, `TEST2`, and `deadbeef`, there _is no symbol_ referenced by the `linkage_name` on the DIE:

```
565d0    LOCAL      OBJECT      _ZN4test5TEST217h8314c5b1b9028ef4E     0x10      .rodata(16)
 73b0    LOCAL      FUNC        _ZN4test8deadbeef17hc8e13bcb4738fc41E  0xf7      .text(14)
565c0    GLOBAL     OBJECT      TEST                                   0x10      .rodata(16)
```

You will note that the symbols include the [symbol hash](https://github.com//m4b/rust/blob/b15a8eafcd7e50af116d398c1ac3c5a0504c0270/src/librustc_trans/back/symbol_names.rs#L152).

In contrast with the cpp version, the symbol name (including parameter types, see the `v` (for void) in `_ZN4test8deadbeefEv`) is identical to the linkage_name, as I think, expected:

```
201040    GLOBAL     OBJECT      TEST                                  0x10    .data(22)
   670    GLOBAL     FUNC        _ZN4test8deadbeefEv                   0x4c    .text(12)
201030    GLOBAL     OBJECT      _ZN4test4TESTE                        0x10    .data(22)
```

# Debuggers

I would now like to present a debugging session (primarily in `gdb`) for the two binaries to attempt to illustrate some of the oddities that occur, and motivate why I think there is something wrong here, at the very least.

There are general ergonomic issues and other oddities that I think are surfacing because of the current debug info situation. I have used `gdb` to illustrate, as `lldb` is essentially non-functioning for me, and when it doesn't segfault, I cannot break on un-mangled names for the rust binaries.

## GDB

First the rust binary:

```
(gdb) ptype TEST
No symbol 'TEST' in current context
(gdb) ptype test::TEST
No symbol 'test::TEST' in current context
(gdb) ptype test::TEST2
Reading in symbols for test.rs...done.
type = struct test::Foo {
  x: u64,
  y: i32,
}
(gdb) ptype test::deadbeef
type = fn ()
(gdb) whatis TEST
No symbol 'TEST' in current context
(gdb) whatis test::TEST
No symbol 'test::TEST' in current context
(gdb) whatis test::TEST2
type = test::Foo
(gdb) whatis test::deadbeef
type = fn ()
(gdb) info addr TEST
Symbol "TEST" is at 0x565c0 in a file compiled without debugging.
(gdb) info addr test::TEST
No symbol "test::TEST" in current context.
(gdb) info addr test::TEST2
Symbol "test::TEST2" is static storage at address 0x565d0.
(gdb) info addr test::deadbeef
Symbol "test::deadbeef" is a function at address 0x73b0.
(gdb) info addr _ZN4test5TEST217h8314c5b1b9028ef4E
Symbol "test::TEST2" is at 0x565d0 in a file compiled without debugging.
(gdb) info addr _ZN4test8deadbeef17hc8e13bcb4738fc41E
Symbol "test::deadbeef" is at 0x73b0 in a file compiled without debugging.
```

The last two are particularly troubling. This certainly looks like a direct consequence of mapping the unmangled, symbol name + hash, in the ELF symbol table to the mangled-without-hash _linkage_ name in the DWARF DIE.

If we compare this to the exact same c++ debugging sequence, it is exactly what one would expect:

```
(gdb) ptype TEST
type = struct test::Foo {
    uint64_t x;
    int64_t y;
}
(gdb) ptype test::TEST
type = struct test::Foo {
    uint64_t x;
    int64_t y;
}
(gdb) ptype test::deadbeef
type = void (void)
(gdb) whatis TEST
type = test::Foo
(gdb) whatis test::TEST
type = test::Foo
(gdb) whatis test::deadbeef
type = void (void)
(gdb) info addr TEST
Symbol "TEST" is static storage at address 0x201040.
(gdb) info addr test::TEST
Symbol "test::TEST" is static storage at address 0x201030.
(gdb) info addr test::deadbeef
Symbol "test::deadbeef()" is a function at address 0x670.
(gdb) info addr _ZN4test4TESTE
Symbol "test::TEST" is static storage at address 0x201030.
(gdb) info addr _ZN4test8deadbeefEv
Symbol "test::deadbeef()" is a function at address 0x670.
```


## LLDB

As of lldb with llvm 5.0.0 I cannot even auto-complete or break on non-mangled names, which further increases my suspicion something is wrong, and the fact that it segfaults occasionally when I auto-complete using `test::` as a seed and the stack trace indicates its in the dwarf DIE parsing logic is equally suspicious:

```
Process 11803 (lldb) of user 1000 dumped core.
Stack trace of thread 11841:
#0  0x00007f4d2e9a4295 _ZN16DWARFCompileUnit6GetDIEEj (liblldb.so.3.8.0)
#1  0x00007f4d2e9a9f60 _ZN14DWARFDebugInfo6GetDIEERK6DIERef (liblldb.so.3.8.0)
#2  0x00007f4d2e9a4265 _ZN16DWARFCompileUnit6GetDIEEj (liblldb.so.3.8.0)
#3  0x00007f4d2e9ac6b0 _ZNK19DWARFDebugInfoEntry13GetAttributesEPK16DWARFCompileUnitN14DWARFFormValue14
#4  0x00007f4d2e9a4f89 _ZN16DWARFCompileUnit12IndexPrivateEPS_N4lldb12LanguageTypeERKN14DWARFFormValue1
#5  0x00007f4d2e9a454c _ZN16DWARFCompileUnit5IndexER9NameToDIES1_S1_S1_S1_S1_S1_S1_ (liblldb.so.3.8.0)
#6  0x00007f4d2e9cab60 _ZNSt17_Function_handlerIFSt10unique_ptrINSt13__future_base12_Result_baseENS2_8_
#7  0x00007f4d2e9caa1a _ZNSt13__future_base13_State_baseV29_M_do_setEPSt8functionIFSt10unique_ptrINS_12
#8  0x00007f4d31feedbf __pthread_once_slow (libpthread.so.0)
#9  0x00007f4d2e9ca5dc _ZNSt13__future_base11_Task_stateISt5_BindIFZN10TaskRunnerIjE7AddTaskIRZN15Symbo
#10 0x00007f4d2ec2fd15 _ZN12_GLOBAL__N_112TaskPoolImpl6WorkerEPS0_ (liblldb.so.3.8.0)
#11 0x00007f4d2c970a6f execute_native_thread_routine (libstdc++.so.6)
#12 0x00007f4d31fe708a start_thread (libpthread.so.0)
#13 0x00007f4d2c3de47f __clone (libc.so.6)
```

# Solutions

I believe 1. is the first that should be attempted, but I am not an expert in the compiler internals.

1. Have the [dwarf name mangler](https://github.com//m4b/rust/blob/b15a8eafcd7e50af116d398c1ac3c5a0504c0270/src/librustc_trans/debuginfo/namespace.rs#L25) include the [symbol hash](https://github.com//m4b/rust/blob/b15a8eafcd7e50af116d398c1ac3c5a0504c0270/src/librustc_trans/back/symbol_names.rs#L152), specifically I think it needs to be appended right about here: https://github.com//m4b/rust/blob/b15a8eafcd7e50af116d398c1ac3c5a0504c0270/src/librustc_trans/debuginfo/namespace.rs#L47
2. Output non-hash versions of symbols into the symbol table that point to the same address, etc., but which reference the non-hash name in the string table.  This is a hack imho, and I've already tried this via binary editing and it did not seem to have any noticeable effect.

Eventually, I think that the final, correct solution imho is to exactly mirror the dwarf output of both the gcc and clang backends, which means:

1. Making the linkage_name = the symbol table name
2. Adding parameters, etc., into the symbol table name?

# Final Considerations

I believe I've also found some weirdness w.r.t. other symbol names, specifically:

1. locally scoped statics,
2. impls of traits with certain annotations

examples of which are:

```
_ZN3std10sys_common9backtrace11log_enabled7ENABLED17hc187c5b3618ccb2eE.0.0
_ZN61_$LT$alloc..heap..Heap$u20$as$u20$alloc..allocator..Alloc$GT$3oom17h28fb525969c57bd8E
```

at lines:

1. https://github.com//m4b/rust/blob/b15a8eafcd7e50af116d398c1ac3c5a0504c0270/src/libstd/sys_common/backtrace.rs#L148 (`0.0` appended to a mangled name is not a valid mangled name afaik)
2. https://github.com//m4b/rust/blob/b15a8eafcd7e50af116d398c1ac3c5a0504c0270/src/liballoc/heap.rs#L94-L100 (the symbol name appears to differ widly from the debug name)

, respectively.

But I think this is another story for another time ;)