Repository: oppia/oppia

Issue:
Build keeps failing due to incomplete webpack compilation
**Describe the bug**
Build fails due to webpack compilation being incomplete.

**To Reproduce**

1. Run `python -m scripts.build --prod_env`
2. Build will fail intermittently with error:

```
Starting new Webpack Compiler: /Users/saxenaankita/Downloads/opensrc/oppia/../oppia_tools/node-16.13.0/bin/node --max-old-space-size=4096 /Users/saxenaankita/Downloads/opensrc/oppia/node_modules/webpack/bin/webpack.js --config webpack.prod.sourcemap.config.ts

<--- Last few GCs --->

[2809:0x7fbca0008000]   565548 ms: Scavenge 3818.4 (3889.9) -> 3817.0 (3902.2) MB, 7.5 / 0.0 ms  (average mu = 0.314, current mu = 0.399) allocation failure 
[2809:0x7fbca0008000]   565592 ms: Scavenge (reduce) 3831.0 (3903.4) -> 3830.9 (3890.2) MB, 6.7 / 0.0 ms  (average mu = 0.314, current mu = 0.399) allocation failure 
[2809:0x7fbca0008000]   565609 ms: Scavenge (reduce) 3833.7 (3890.2) -> 3832.3 (3891.9) MB, 9.1 / 0.0 ms  (average mu = 0.314, current mu = 0.399) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x103b0e515 node::Abort() (.cold.1) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 2: 0x10280f989 node::Abort() [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 3: 0x10280faff node::OnFatalError(char const*, char const*) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 4: 0x10298f2c7 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 5: 0x10298f263 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 6: 0x102b30975 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 7: 0x102b349bd v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 8: 0x102b3129d v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 9: 0x102b2e7bd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
10: 0x102b2d4d8 v8::internal::Heap::HandleGCRequest() [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
11: 0x102ad9cd1 v8::internal::StackGuard::HandleInterrupts() [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
12: 0x102ebdedc v8::internal::Runtime_StackGuardWithGap(int, unsigned long*, v8::internal::Isolate*) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
13: 0x103267dd9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
14: 0x1133f9036 
Stopping Webpack Compiler(pid=2809)...
Traceback (most recent call last):
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/saxenaankita/Downloads/opensrc/oppia/scripts/build.py", line 1432, in <module>
    main()
  File "/Users/saxenaankita/Downloads/opensrc/oppia/scripts/build.py", line 1419, in main
    build_using_webpack(WEBPACK_PROD_SOURCE_MAPS_CONFIG)
  File "/Users/saxenaankita/Downloads/opensrc/oppia/scripts/build.py", line 690, in build_using_webpack
    'webpack_bundles should be non-empty.')
AssertionError: webpack_bundles should be non-empty.
```

**Observed behavior**
JavaScript heap out of memory while running webpack compiler

**Expected behavior**
Webpack compilation should work as expected.


**Desktop (please complete the following information; delete this section if the issue does not arise on desktop):**
 - OS: MacOs

Build keeps failing due to incomplete webpack compilation
**Describe the bug**
Build fails due to webpack compilation being incomplete.

**To Reproduce**

1. Run `python -m scripts.build --prod_env`
2. Build will fail intermittently with error:

```
Starting new Webpack Compiler: /Users/saxenaankita/Downloads/opensrc/oppia/../oppia_tools/node-16.13.0/bin/node --max-old-space-size=4096 /Users/saxenaankita/Downloads/opensrc/oppia/node_modules/webpack/bin/webpack.js --config webpack.prod.sourcemap.config.ts

<--- Last few GCs --->

[2809:0x7fbca0008000]   565548 ms: Scavenge 3818.4 (3889.9) -> 3817.0 (3902.2) MB, 7.5 / 0.0 ms  (average mu = 0.314, current mu = 0.399) allocation failure 
[2809:0x7fbca0008000]   565592 ms: Scavenge (reduce) 3831.0 (3903.4) -> 3830.9 (3890.2) MB, 6.7 / 0.0 ms  (average mu = 0.314, current mu = 0.399) allocation failure 
[2809:0x7fbca0008000]   565609 ms: Scavenge (reduce) 3833.7 (3890.2) -> 3832.3 (3891.9) MB, 9.1 / 0.0 ms  (average mu = 0.314, current mu = 0.399) allocation failure 


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x103b0e515 node::Abort() (.cold.1) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 2: 0x10280f989 node::Abort() [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 3: 0x10280faff node::OnFatalError(char const*, char const*) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 4: 0x10298f2c7 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 5: 0x10298f263 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 6: 0x102b30975 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 7: 0x102b349bd v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 8: 0x102b3129d v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
 9: 0x102b2e7bd v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
10: 0x102b2d4d8 v8::internal::Heap::HandleGCRequest() [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
11: 0x102ad9cd1 v8::internal::StackGuard::HandleInterrupts() [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
12: 0x102ebdedc v8::internal::Runtime_StackGuardWithGap(int, unsigned long*, v8::internal::Isolate*) [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
13: 0x103267dd9 Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_NoBuiltinExit [/Users/saxenaankita/Downloads/opensrc/oppia_tools/node-16.13.0/bin/node]
14: 0x1133f9036 
Stopping Webpack Compiler(pid=2809)...
Traceback (most recent call last):
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/saxenaankita/Downloads/opensrc/oppia/scripts/build.py", line 1432, in <module>
    main()
  File "/Users/saxenaankita/Downloads/opensrc/oppia/scripts/build.py", line 1419, in main
    build_using_webpack(WEBPACK_PROD_SOURCE_MAPS_CONFIG)
  File "/Users/saxenaankita/Downloads/opensrc/oppia/scripts/build.py", line 690, in build_using_webpack
    'webpack_bundles should be non-empty.')
AssertionError: webpack_bundles should be non-empty.
```

**Observed behavior**
JavaScript heap out of memory while running webpack compiler

**Expected behavior**
Webpack compilation should work as expected.


**Desktop (please complete the following information; delete this section if the issue does not arise on desktop):**
 - OS: MacOs



Discussion:
@vojtechjelinek can you please look into this?

I am trying a suggestion by Sean to increase the size [here](https://github.com/oppia/oppia/blob/develop/scripts/build.py#L686) and will let you know if it works. 
@ankita240796 Yep, it should be increased to `6144`, I wanted to prepare a PR today, should I do it?
@vojtechjelinek yes please!
@vojtechjelinek can you please look into this?

I am trying a suggestion by Sean to increase the size [here](https://github.com/oppia/oppia/blob/develop/scripts/build.py#L686) and will let you know if it works. 
@ankita240796 Yep, it should be increased to `6144`, I wanted to prepare a PR today, should I do it?
@vojtechjelinek yes please!

Resolving patch:
diff --git a/scripts/build.py b/scripts/build.py
--- a/scripts/build.py
+++ b/scripts/build.py
@@ -683,7 +683,7 @@ def build_using_webpack(config_path: str) -> None:
 
     print('Building webpack')
     managed_webpack_compiler = servers.managed_webpack_compiler(
-        config_path=config_path, max_old_space_size=4096)
+        config_path=config_path, max_old_space_size=6144)
     with managed_webpack_compiler as p:
         p.wait()
     assert get_file_count('backend_prod_files/webpack_bundles/') > 0, (


Test patch:
diff --git a/scripts/build_test.py b/scripts/build_test.py
--- a/scripts/build_test.py
+++ b/scripts/build_test.py
@@ -1189,7 +1189,7 @@ def mock_managed_webpack_compiler(
             max_old_space_size: int
         ) -> Iterator[scripts_test_utils.PopenStub]:
             self.assertEqual(config_path, build.WEBPACK_PROD_CONFIG)
-            self.assertEqual(max_old_space_size, 4096)
+            self.assertEqual(max_old_space_size, 6144)
             yield scripts_test_utils.PopenStub()
 
         def mock_get_file_count(unused_path: str) -> int:
@@ -1213,7 +1213,7 @@ def mock_managed_webpack_compiler(
             config_path: str, max_old_space_size: int
         ) -> Iterator[scripts_test_utils.PopenStub]:
             self.assertEqual(config_path, build.WEBPACK_PROD_CONFIG)
-            self.assertEqual(max_old_space_size, 4096)
+            self.assertEqual(max_old_space_size, 6144)
             yield scripts_test_utils.PopenStub()
 
         def mock_get_file_count(unused_path: str) -> int: