Repository: neovim
Issue: lua-cjson crash after opening large log file with LSP and/or Copilot
Labels: has:repro,has:backtrace,bug-crash,lsp,dependencies

### Problem

This is very similar to #29037 and might be the same.

I was opening a 587MB restic log file and after a few seconds neovim would crash with one of these:
1. `realloc(): invalid next size`
2. `malloc(): invalid next size (unsorted)`
3. `corrupted size vs. prev_size`

there might be more.

Disabling the `copilot.lua` plugin does seem to circumvent the issue.

Unfortunately it seems this depends a lot on my full setup and the stacktraces seem to have gaps in them.
A `find / -xdev -ls > large_file` + 'nvim large_file` seems to also trigger the issue, maybe it will on other machines also.

Maybe this is just a luajit issue?

What I can provide next or do to help?


Details:
neovim from Arch (extra) with astronvim and a bunch of plugins.

```
NVIM v0.10.1
Build type: Release
LuaJIT 2.1.1723675123
```

Backtraces:
1.`realloc(): invalid next size`
```
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007633cdcae463 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x00007633cdc55120 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007633cdc3c4c3 in __GI_abort () at abort.c:79
#4  0x00007633cdc3d354 in __libc_message_impl (fmt=fmt@entry=0x7633cddcb2f5 "%s\n") at ../sysdeps/posix/libc_fatal.c:132
#5  0x00007633cdcb8765 in malloc_printerr (str=str@entry=0x7633cddc9169 "realloc(): invalid next size") at malloc.c:5772
#6  0x00007633cdcbca04 in _int_realloc (av=av@entry=0x7633cddffac0 <main_arena>, oldp=oldp@entry=0x5a2f33193650, oldsize=oldsize@entry=8192, nb=nb@entry=16384) at malloc.c:4939
#7  0x00007633cdcbda15 in __GI___libc_realloc (oldmem=0x5a2f33193660, bytes=16368) at malloc.c:3508
#8  0x00005a2ede3f3b9c in ?? ()
#9  0x00005a2ede3f4470 in ?? ()
#10 0x00005a2ede3f4470 in ?? ()
#11 0x00005a2ede3f4470 in ?? ()
#12 0x00005a2ede3f4b48 in ?? ()
#13 0x00007633cdf60f06 in lj_BC_FUNCC () at buildvm_x86.dasc:857
#14 0x00007633cdf7512a in lua_pcall (L=0x7633ce096380, nargs=<optimized out>, nresults=0, errfunc=<optimized out>) at /usr/src/debug/luajit/LuaJIT-ae4735f621d89d84758769b76432d2319dda9827/src/lj_api.c:1122
#15 0x00005a2ede1e8e08 in ?? ()
#16 0x00005a2ede35af18 in state_handle_k_event ()
#17 0x00005a2ede270ae1 in ?? ()
#18 0x00005a2ede260b67 in ?? ()
#19 0x00005a2ede35acc3 in state_enter ()
#20 0x00005a2ede25dd2a in normal_enter ()
#21 0x00005a2eddff7b27 in main ()
```

2. `malloc(): invalid next size (unsorted)`
```
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007131068c9463 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x0000713106870120 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007131068574c3 in __GI_abort () at abort.c:79
#4  0x0000713106858354 in __libc_message_impl (fmt=fmt@entry=0x7131069e62f5 "%s\n") at ../sysdeps/posix/libc_fatal.c:132
#5  0x00007131068d3765 in malloc_printerr (str=str@entry=0x7131069e9908 "malloc(): invalid next size (unsorted)") at malloc.c:5772
#6  0x00007131068d6d2c in _int_malloc (av=av@entry=0x713106a1aac0 <main_arena>, bytes=bytes@entry=16369) at malloc.c:4081
#7  0x00007131068d7902 in _int_realloc (av=av@entry=0x713106a1aac0 <main_arena>, oldp=oldp@entry=0x5b6b7a2a8d80, oldsize=oldsize@entry=8192, nb=nb@entry=16384) at malloc.c:4975
#8  0x00007131068d8a15 in __GI___libc_realloc (oldmem=0x5b6b7a2a8d90, bytes=16368) at malloc.c:3508
#9  0x00005b6b3a2b5b9c in ?? ()
#10 0x00005b6b3a2b6470 in ?? ()
#11 0x00005b6b3a2b6470 in ?? ()
#12 0x00005b6b3a2b6470 in ?? ()
#13 0x00005b6b3a2b6b48 in ?? ()
#14 0x0000713106b7bf06 in lj_BC_FUNCC () at buildvm_x86.dasc:857
#15 0x0000713106b9012a in lua_pcall (L=0x713106cb1380, nargs=<optimized out>, nresults=0, errfunc=<optimized out>)
    at /usr/src/debug/luajit/LuaJIT-ae4735f621d89d84758769b76432d2319dda9827/src/lj_api.c:1122
#16 0x00005b6b3a0aae08 in ?? ()
#17 0x00005b6b3a21cf18 in state_handle_k_event ()
#18 0x00005b6b3a132ae1 in ?? ()
#19 0x00005b6b3a122b67 in ?? ()
#20 0x00005b6b3a21ccc3 in state_enter ()
#21 0x00005b6b3a11fd2a in normal_enter ()
#22 0x00005b6b39eb9b27 in main ()
```

3.`corrupted size vs. prev_size`

```
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007d2f37278463 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x00007d2f3721f120 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007d2f372064c3 in __GI_abort () at abort.c:79
#4  0x00007d2f37207354 in __libc_message_impl (fmt=fmt@entry=0x7d2f373952f5 "%s\n") at ../sysdeps/posix/libc_fatal.c:132
#5  0x00007d2f37282765 in malloc_printerr (str=str@entry=0x7d2f37392ff8 "corrupted size vs. prev_size") at malloc.c:5772
#6  0x00007d2f372833a6 in unlink_chunk (p=p@entry=0x5fcc0a155fd0, av=0x7d2f373c9ac0 <main_arena>) at malloc.c:1611
#7  0x00007d2f372869b8 in _int_realloc (av=av@entry=0x7d2f373c9ac0 <main_arena>, oldp=oldp@entry=0x5fcc0a153fd0, oldsize=oldsize@entry=8192, nb=nb@entry=16384) at malloc.c:4969
#8  0x00007d2f37287a15 in __GI___libc_realloc (oldmem=0x5fcc0a153fe0, bytes=16368) at malloc.c:3508
#9  0x00005fcbc5375b9c in ?? ()
#10 0x00005fcbc5376470 in ?? ()
#11 0x00005fcbc5376470 in ?? ()
#12 0x00005fcbc5376470 in ?? ()
#13 0x00005fcbc5376b48 in ?? ()
#14 0x00007d2f3752af06 in lj_BC_FUNCC () at buildvm_x86.dasc:857
#15 0x00007d2f3753f12a in lua_pcall (L=0x7d2f37660380, nargs=<optimized out>, nresults=0, errfunc=<optimized out>) at /usr/src/debug/luajit/LuaJIT-ae4735f621d89d84758769b76432d2319dda9827/src/lj_api.c:1122
#16 0x00005fcbc516ae08 in ?? ()
#17 0x00005fcbc52dcf18 in state_handle_k_event ()
#18 0x00005fcbc51f2ae1 in ?? ()
#19 0x00005fcbc51e2b67 in ?? ()
#20 0x00005fcbc52dccc3 in state_enter ()
#21 0x00005fcbc51dfd2a in normal_enter ()
#22 0x00005fcbc4f79b27 in main ()
```

### Steps to reproduce

1. Open a large text file, somewhere in the range of > 400MB
2. `:lua a = vim.json.encode(table.concat(vim.api.nvim_buf_get_lines(0, 1, -1, false), "\n"))`

### Expected behavior

no crash.

### Neovim version (nvim -v)

v0.10.1

### Vim (not Nvim) behaves the same?

unknown

### Operating system/version

Arch

### Terminal name/version

wezterm 20240722-080956-7e8fdc11

### $TERM environment variable

xterm-256color

### Installation

Arch (extra)