Repository: rust
Issue: Reference to a #[thread_local] is dereferenced too early
Labels: O-macos,T-compiler,A-thread-locals,C-bug

I'm working on the [dynamic library for elfmalloc](https://github.com/ezrosent/allocators-rs/tree/master/elfc) for Mac, and one of the caveats that we have to work with is that, during dynamic library loading, TLS is unavailable (if you try to access a `#[thread_local]`, the program simply aborts).

In order to work around this, we have a mechanism which checks to make sure that dynamic loading is finished before we access any thread-local storage (see the [`with` method](https://github.com/ezrosent/allocators-rs/blob/6f85313c51f42f0b4f8ea7d6d938caa5eab54412/alloc-tls/src/lib.rs#L209)).

Recently, we encountered a strange bug: We were spuriously accessing thread-local values while dynamic loading was still happening. The code in question looks like this:

```rust
pub unsafe fn alloc(size: usize) -> *mut u8 {
    LOCAL_ELF_HEAP.with(|h| (*h.get()).alloc.alloc(size))
        .unwrap_or_else(|| super::large_alloc::alloc(size))
}
```

`LOCAL_ELF_HEAP` is a `#[thread_local]` variable, and in the `with` method, we make sure that dynamic loading is done before we dereference `&self`. To make a long story short, it appears as though the generated code has reordered the instructions to dereference `LOCAL_ELF_HEAP` *before* we check to make sure that dynamic loading is done. You can see that in the disassembly of the `alloc` method (output from lldb):

```text
    0x10f1b7670 <+0>:  pushq  %rbp
    0x10f1b7671 <+1>:  movq   %rsp, %rbp
    0x10f1b7674 <+4>:  subq   $0x40, %rsp
    0x10f1b7678 <+8>:  leaq   -0x20(%rbp), %rax
    0x10f1b767c <+12>: leaq   -0x28(%rbp), %rcx
    0x10f1b7680 <+16>: movq   %rdi, -0x28(%rbp)
    0x10f1b7684 <+20>: movq   %rcx, -0x10(%rbp)
    0x10f1b7688 <+24>: movq   -0x10(%rbp), %rdx
    0x10f1b768c <+28>: leaq   0xaf5f5(%rip), %rdi       ; elfmalloc::general::global::LOCAL_ELF_HEAP::h7d8e035078da16c9
    0x10f1b7693 <+35>: movq   %rax, -0x30(%rbp)
    0x10f1b7697 <+39>: movq   %rdx, -0x38(%rbp)
    0x10f1b769b <+43>: callq  *(%rdi)
->  0x10f1b769d <+45>: leaq   -0x20(%rbp), %rdi
    0x10f1b76a1 <+49>: movq   %rax, %rsi
    0x10f1b76a4 <+52>: movq   -0x38(%rbp), %rdx
    0x10f1b76a8 <+56>: callq  0x10f1e2ac0               ; _$LT$alloc_tls..TLSSlot$LT$T$GT$$GT$::with::h2a59e6dd7499a672 at lib.rs:209
    0x10f1b76ad <+61>: leaq   -0x20(%rbp), %rdi
    0x10f1b76b1 <+65>: leaq   -0x28(%rbp), %rax
    0x10f1b76b5 <+69>: movq   %rax, -0x8(%rbp)
    0x10f1b76b9 <+73>: movq   -0x8(%rbp), %rsi
    0x10f1b76bd <+77>: callq  0x10f1eb9a0               ; _$LT$core..option..Option$LT$T$GT$$GT$::unwrap_or_else::h09d8edc34df58ca5 at option.rs:373
    0x10f1b76c2 <+82>: movq   %rax, -0x40(%rbp)
    0x10f1b76c6 <+86>: movq   -0x40(%rbp), %rax
    0x10f1b76ca <+90>: addq   $0x40, %rsp
    0x10f1b76ce <+94>: popq   %rbp
    0x10f1b76cf <+95>: retq
```

The `->` points to where the next instruction to be executed at the time that the program aborted. The `callq` is to `libdyld.dylib`'s `_tlv_bootstrap`, which is simply [implemented as](https://opensource.apple.com/source/dyld/dyld-239.3/src/threadLocalVariables.c) `abort();`.

The key thing to note is that the call to `_tlv_boostrap` (which attempts - and fails - to access the `#[thread_local]`) is *before* the call to `with` a couple of instructions later. I'm not positive, but my hypothesis is that the compiler reordered the accesses because immutable references are supposed to guarantee that the referent exists, and so it thought that the reordering was valid.