Repository: node
Issue: parallel/test-worker-nearheaplimit-deadlock is flaky
Labels: flaky-test

Refs: https://github.com/nodejs/reliability/issues/720

It seems the failure started about 3 days ago, around the time that https://github.com/nodejs/node/pull/48655 landed. I cannot reproduce it locally however.

One peculiar bit I noticed from the error logs:

```
16:17:07      1: 0x104c0fb6c node::Abort() [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      2: 0x104c0fd6c node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      3: 0x104daf0fc v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      4: 0x104f8fb50 v8::internal::Heap::stack() [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      5: 0x104fab008 v8::internal::LocalFactory::NumberToStringCacheHash(v8::internal::Tagged<v8::internal::Smi>) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
```

<details>
<summary> See full stack trace </summary>

```
16:17:07 not ok 3261 parallel/test-worker-nearheaplimit-deadlock
16:17:07   ---
16:17:07   duration_ms: 193.09600
16:17:07   severity: crashed
16:17:07   exitcode: -6
16:17:07   stack: |-
16:17:07     
16:17:07     <--- Last few GCs --->
16:17:07     
16:17:07     [15425:0x128098000]       20 ms: Mark-Compact 1.7 (6.0) -> 1.7 (6.0) MB, 1.12 / 0.00 ms  (average mu = 0.312, current mu = 0.000) background allocation failure; GC in old space requested
16:17:07     [15425:0x128098000]       21 ms: Mark-Compact 1.7 (6.0) -> 1.7 (6.0) MB, 1.42 / 0.00 ms  (average mu = 0.208, current mu = 0.055) background allocation failure; GC in old space requested
16:17:07     
16:17:07     
16:17:07     <--- JS stacktrace --->
16:17:07     
16:17:07     FATAL ERROR: LocalHeap: allocation failed Allocation failed - JavaScript heap out of memory
16:17:07     ----- Native stack trace -----
16:17:07     
16:17:07      1: 0x104c0fb6c node::Abort() [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      2: 0x104c0fd6c node::ModifyCodeGenerationFromStrings(v8::Local<v8::Context>, v8::Local<v8::Value>, bool) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      3: 0x104daf0fc v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      4: 0x104f8fb50 v8::internal::Heap::stack() [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      5: 0x104fab008 v8::internal::LocalFactory::NumberToStringCacheHash(v8::internal::Tagged<v8::internal::Smi>) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      6: 0x104f641c8 v8::internal::FactoryBase<v8::internal::LocalFactory>::NewByteArray(int, v8::internal::AllocationType) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      7: 0x104e01058 v8::internal::Handle<v8::internal::ByteArray> v8::internal::baseline::BytecodeOffsetTableBuilder::ToBytecodeOffsetTable<v8::internal::LocalIsolate>(v8::internal::LocalIsolate*) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      8: 0x104e00f2c v8::internal::baseline::BaselineCompiler::Build(v8::internal::LocalIsolate*) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07      9: 0x104e17050 v8::internal::GenerateBaselineCode(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SharedFunctionInfo>) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     10: 0x104e657e8 v8::internal::Compiler::CompileSharedWithBaseline(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SharedFunctionInfo>, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     11: 0x104e65a9c v8::internal::Compiler::CompileBaseline(v8::internal::Isolate*, v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ClearExceptionFlag, v8::internal::IsCompiledScope*) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     12: 0x104dfaf3c v8::internal::baseline::BaselineBatchCompiler::EnqueueFunction(v8::internal::Handle<v8::internal::JSFunction>) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     13: 0x104f1ce68 v8::internal::TieringManager::OnInterruptTick(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::CodeKind) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     14: 0x105338bd0 v8::internal::(anonymous namespace)::BytecodeBudgetInterruptWithStackCheck(v8::internal::Isolate*, v8::internal::Arguments<(v8::internal::ArgumentsType)0>&, v8::internal::CodeKind) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     15: 0x105337694 v8::internal::Runtime_BytecodeBudgetInterruptWithStackCheck_Ignition(int, unsigned long*, v8::internal::Isolate*) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     16: 0x1056d3954 Builtins_CEntry_Return1_ArgvOnStack_NoBuiltinExit [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     17: 0x1057aa5c4 Builtins_JumpLoopHandler [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     18: 0x105648b84 Builtins_InterpreterEntryTrampoline [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     19: 0x15000bb84 
16:17:07     20: 0x15000ce8c 
16:17:07     21: 0x105648b84 Builtins_InterpreterEntryTrampoline [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     22: 0x105648b84 Builtins_InterpreterEntryTrampoline [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     23: 0x1056468ac Builtins_JSEntryTrampoline [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     24: 0x105646594 Builtins_JSEntry [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     25: 0x104ef1740 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     26: 0x104ef0cc0 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     27: 0x104dc8b28 v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     28: 0x104bf2a50 node::builtins::BuiltinLoader::CompileAndCall(v8::Local<v8::Context>, char const*, node::Realm*) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     29: 0x104c822d4 node::PrincipalRealm::BootstrapRealm() [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     30: 0x104c80298 node::Realm::RunBootstrapping() [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     31: 0x104b40d98 node::CreateEnvironment(node::IsolateData*, v8::Local<v8::Context>, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&, node::EnvironmentFlags::Flags, node::ThreadId, std::__1::unique_ptr<node::InspectorParentHandle, std::__1::default_delete<node::InspectorParentHandle> >) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     32: 0x104cc486c node::worker::Worker::Run() [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     33: 0x104cc7f50 node::worker::Worker::StartThread(v8::FunctionCallbackInfo<v8::Value> const&)::$_3::__invoke(void*) [/Users/iojs/build/workspace/node-test-commit-osx-arm/nodes/osx11/out/Release/node]
16:17:07     34: 0x181467878 _pthread_start [/usr/lib/system/libsystem_pthread.dylib]
16:17:07     35: 0x1814625e0 thread_start [/usr/lib/system/libsystem_pthread.dylib]
16:17:07   ...
```
</details>

It seems strange that the ModifyCodeGenerationFromStrings callback can get called from FatalProcessOutOfMemory. Even if this gets called for OOM somehow, that shouldn't do much unless source maps are enabled, in which case it would call into JavaScript and that's definitely going to crash in OOM.