Repository: pytorch
Issue: Segfault linear_search_fdes when calling c10::get_backtrace 
Labels: module: internals,triaged,has workaround

### 🐛 Describe the bug

My OSS PyTorch build on devgpu has started segfaulting when collecting backtraces. Segfault looks like

```
linear_search_fdes (ob=ob@entry=0x7fffffff90c0, this_fde=0x0, pc=pc@entry=0x7ffff7d5ab12 <backtrace+98>)
    at ../../../libgcc/unwind-dw2-fde.c:969
969     ../../../libgcc/unwind-dw2-fde.c: No such file or directory.
(gdb) bt
#0  linear_search_fdes (ob=ob@entry=0x7fffffff90c0, this_fde=0x0, pc=pc@entry=0x7ffff7d5ab12 <backtrace+98>)
    at ../../../libgcc/unwind-dw2-fde.c:969
#1  0x00007ffff610108a in find_fde_tail (dbase=8,
    bases=<error reading variable: dwarf2_find_location_expression: Corrupted DWARF expression.>, hdr=0x7ffff7dc37bc,
    pc=<error reading variable: dwarf2_find_location_expression: Corrupted DWARF expression.>)
    at ../../../libgcc/unwind-dw2-fde-dip.c:519
#2  _Unwind_Find_FDE (pc=<optimized out>, bases=bases@entry=0x7fffffff9238) at ../../../libgcc/unwind-dw2-fde-dip.c:573
#3  0x00007ffff60fdbcd in uw_frame_state_for (context=context@entry=0x7fffffff9190, fs=fs@entry=0x7fffffff9280)
    at ../../../libgcc/unwind-dw2.c:1005
#4  0x00007ffff60fef65 in _Unwind_Backtrace (trace=0x7ffff7d5aa10 <backtrace_helper>, trace_argument=0x7fffffff93c0)
    at ../../../libgcc/unwind.inc:303
#5  0x00007ffff7d5ab13 in backtrace () from /lib64/libc.so.6
#6  0x00007fffefd973ff in c10::get_backtrace[abi:cxx11](unsigned long, unsigned long, bool) ()
   from /data/users/ezyang/a/pytorch/torch/lib/libc10.so
#7  0x00007fffefd9adc9 in std::_Function_handler<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > (), c10::(anonymous namespace)::GetFetchStackTrace()::{lambda()#1}>::_M_invoke(std::_Any_data const&) ()
   from /data/users/ezyang/a/pytorch/torch/lib/libc10.so
#8  0x00007fffefd9c695 in c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) () from /data/users/ezyang/a/pytorch/torch/lib/libc10.so
#9  0x00007fffefd4cb20 in c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /data/users/ezyang/a/pytorch/torch/lib/libc10.so
#10 0x00007fffe4260110 in c10::impl::common_device_check_failure(c10::Device, at::Tensor const&, char const*, char const*) ()
   from /data/users/ezyang/a/pytorch/torch/lib/libtorch_cpu.so
#11 0x00007fffd89aee09 in at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA__index_select(at::Tensor const&, long, at::Tensor const&) () from /data/users/ezyang/a/pytorch/torch/lib/libtorch_cuda.so
#12 0x00007fffd89aeec3 in c10::impl::wrap_kernel_functor_unboxed_<c10::impl::detail::WrapFunctionIntoFunctor_<c10::CompileTimeFunctionPointer<at::Tensor (at::Tensor const&, long, at::Tensor const&), &at::(anonymous namespace)::(anonymous namespace)::wrapper_CUDA__index_select>, at::Tensor, c10::guts::typelist::typelist<at::Tensor const&, long, at::Tensor const&> >, at::Tensor (at::Tensor const&, long, at::Tensor const&)>::call(c10::OperatorKernel*, c10::DispatchKeySet, at::Tensor const&, long, at::Tensor const&) ()
   from /data/users/ezyang/a/pytorch/torch/lib/libtorch_cuda.so
```

Not sure how easy it is to reproduce. Not sure what could have changed to cause this to fail lol.  Posting this for Google juice.

### Versions

main

cc @gchanan @zou3519 @kadeng @bhosmer @smessmer @ljk53 @bdhirsh