Repository: go
Issue: proposal: cmd/link: by default, do not write out DWARF
Labels: Proposal,Proposal-Hold

This is not as radical as it sounds.

At the very least, we need to understand what the default should be when building the Go installation: write out DWARF, or not? The costs and benefits are more subtle than some realize.

Update: As demonstrated below, dropping DWARF also causes a significant improvement in build/install time.

```
% cat hello.go
package main

import "fmt"

func main() {
	fmt.Printf("hello world")
}
```

With this canonical, trivial, but also representative program as input, I used a sequence of Go versions to build the binary on my Mac (OSX 10.13.5, amd64). I have sorted the list into chronological order by version:

```
% ls -l hello*
-rw-r--r--+   1 r  staff          71 Apr  8  2014 hello.go
-rwxr-xr-x    1 r  staff     1919504 Jun 27 13:44 hello1.4 # built with Go 1.4
-rwxr-xr-x    1 r  staff     1616000 Jun 27 13:54 hello1.7 # built with Go 1.7
-rwxr-xr-x    1 r  staff     1632480 Jun 27 13:45 hello1.8 # built with Go 1.8
-rwxr-xr-x    1 r  staff     1941456 Jun 27 13:47 hello1.9 # built with Go 1.9
-rwxr-xr-x    1 r  staff     2106672 Jun 27 13:50 hello1.10 # built with Go 1.10
-rwxr-xr-x    1 r  staff     2964464 Jun 27 14:01 hello-21Jun-2018 # built with Go at tip on 21 June 2018 - this is just before DWARF compression went in
-rwxr-xr-x    1 r  staff     1970552 Jun 27 13:53 hello1.11beta1 # built with Go 1.11 beta 1 # built with Go 1.11 beta 1, with DWARF compression
```

I believe the drop from 1.4 to 1.7 (1.5 and 1.6 won't run on my Mac any more) is due to various cleanups in the binary triggered by https://github.com/golang/go/issues/6853.

The growth after that is pretty much all due to DWARF. Absent compression, DWARF debugging is now half the binary, as reported by @rsc's sizecmp:

```
% sizecmp hello1.7 hello-21Jun-2018 
__bss                               108784      116464       +7680
__data                                6144       26896      +20752
__debug_abbrev                         255         467        +212
__debug_aranges                         48           0         -48
__debug_frame                        68836       81036      +12200
__debug_gdb_scri                        40          40          +0
__debug_info                        245056      482436     +237380
__debug_line                        101426      146608      +45182
__debug_loc                              0      436989     +436989
__debug_pubnames                     61781       32854      -28927
__debug_pubtypes                     26794       44637      +17843
__debug_ranges                           0      153520     +153520
__gopclntab                         277492      478600     +201108
__gosymtab                               0           0          +0
__itablink                              64          96         +32
__nl_symbol_ptr                          0         144        +144
__noptrbss                           19520        9208      -10312
__noptrdata                           8264       52284      +44020
__rodata                            208087      290543      +82456
__symbol_stub1                           0         108        +108
__text                              508560      592476      +83916
__typelink                            2732        3032        +300
                         total     1643883     2948438    +1304555
%
```

That's 1.3MB of growth, almost all in debug info. Even the PC-to-line table grew massively, quite disproportionate to text size, which is inexplicable to me, but also a bit off topic.

So, DWARF is huge, but we need it, right?

I don't think we do, most of the time. Surely when we are using Delve or GDB or perhaps one day LLDB, yes, but mostly not.

The need for DWARF and other debugging support in Go programs is much less than the corresponding need in C programs, for which DWARF was designed. Go binaries already include basic type information (reflection), a simple symbol table, and PC-to-line data. These not only help the running program, they also provide valuable debugging aids as they stand.

Even without DWARF at all, stack traces cased by panic would be unchanged and would contain symbols and line numbers. Pprof, objdump, and many other tools would still work.

_The DWARF tables are present only for the debuggers._

And useful though the debuggers are sometimes, they are not used often and often not used at all. I think Delve is a great tool, but I use it only once or twice a year because the existing, built-in debugging information is almost always all I need. Why then do we write bloated binaries, paying a cost in file I/O and DWARF write time (not to mention time to compress now) when _half the data in the binary is almost never used?_

If I look in my personal bin directory, it consists of a few shell scripts and many Go binaries, and the net size is in the gigabytes. Gigabytes of binaries! I could delete the DWARF data from all of them and get much of the space back at no cost.

Also, keep in mind that much of this data is redundant. Yes, the addresses change between binaries but the type information that we write out for the runtime, garbage collector, and so on is a megabyte or more of utter redundancy, unvarying yet unshared.

The counterargument to dropping DWARF is of course that people want to debug their programs. The recent Go developer survey reported much higher concern for good debugging support than for reducing binary size. But I stress, most programs are never shown to a debugger, and many programmers only rarely use a debugger on a Go binary.

The desire to have good debugging does not immediately translate into writing out full massive DWARF data every time we build and install a program. (Test binaries and such actually skip DWARF, by default.) I believe time might be better spent improving native debugging support in the binaries, such as more informative stack traces, but that is another topic.

So to the proposal itself:

I propose we change the Go build environment to suppress DWARF by default, saving lots of CPU time and disk space. Instead, a global shell environment variable, say GODWARF=1, could be set to cause it to be written out. Programmers that want DWARF can set that once, in their shell profile, and have full data available. Others could set it only occasionally, on bad days.

For the rest of us, the rest of the time, why bother with it?

If it is decided that DWARF is too valuable to disable by default, I would instead propose a variant of this proposal, where I could set GOWARF=0 and turn it off in perpetuity.

In other words, I am proposing two things.

1) Provide a mechanism, such as a global shell environment variable, to control whether DWARF is written by the tool chain.

2) Decide whether that setting should switch to "no DWARF" by default. I would like that, but would be almost as happy just to have part 1: a simple way to suppress it.

Note: It's not easy enough to use `-ldflags=-w`, since there is no mechanism to set LD flags globally in the Go toolchain. Perhaps that's another way to approach the problem.