Repository: spack/spack

Issue:
Python quit unexpectedly, Python 3.8.0 on macOS Catalina
I'm getting the strangest error message. When I build Python 3.8.0 on macOS Catalina, everything builds fine, and all the libraries appear to be linked correctly, but when I try to build something like numpy or scipy that depends on Python, I get one of those "would you like to report this bug to the developers" pop-ups that says "Python quit unexpectedly".

### Steps to reproduce the issue

```console
$ spack install --test=root py-numpy ^python@3:
$ spack install py-scipy ^python@3:
```

### Error Message

The strange thing is that it says Python 3.7.3 crashed (this is the system Python that comes with macOS that I'm using to run Spack), and the Parent Process is `python3.8`, which is the executable I'm using to build numpy/scipy. I'm not sure why Python 3.8 would be calling Python 3.7.

```
Process:               Python [2113]
Path:                  /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.7.3 (3.7.3)
Build Info:            python3-45000000000000~369
Code Type:             X86-64 (Native)
Parent Process:        python3.8 [1049]
Responsible:           Terminal [613]
User ID:               501

Date/Time:             2019-10-17 18:21:21.268 -0500
OS Version:            Mac OS X 10.15 (19A602)
Report Version:        12
Anonymous UUID:        F81945A4-B77A-7472-CE03-3264A0A13B2D

Sleep/Wake UUID:       9680B008-DDB1-4652-B6F6-96C5AAFF7883

Time Awake Since Boot: 23000 seconds
Time Since Wake:       21000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Abort trap: 6
Termination Reason:    Namespace SIGNAL, Code 0x6

Application Specific Information:
dyld2 mode

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff66423ade __abort_with_payload + 10
1   libsystem_kernel.dylib        	0x00007fff664254ef abort_with_payload_wrapper_internal + 80
2   libsystem_kernel.dylib        	0x00007fff6642549f abort_with_reason + 19
3   org.python.python3            	0x0000000100d23f05 fatal_error.cold.1 + 21
4   org.python.python3            	0x0000000100ca6f78 fatal_error + 536
5   org.python.python3            	0x0000000100ca643e _Py_FatalInitError + 30
6   org.python.python3            	0x0000000100cd3158 pymain_main + 8936
7   org.python.python3            	0x0000000100cd33bf _Py_UnixMain + 111
8   libdyld.dylib                 	0x00007fff662bb405 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000002000209  rbx: 0x0000000000000002  rcx: 0x00007ffeef086748  rdx: 0x0000000000000000
  rdi: 0x0000000000000002  rsi: 0x0000000000000006  rbp: 0x00007ffeef086790  rsp: 0x00007ffeef086748
   r8: 0x00007fcc96800e00   r9: 0x0000000000000002  r10: 0x0000000000000000  r11: 0x0000000000000246
  r12: 0x0000000000000000  r13: 0x0000000000000000  r14: 0x0000000000000006  r15: 0x0000000000000002
  rip: 0x00007fff66423ade  rfl: 0x0000000000000246  cr2: 0x0000000100fc7020
  
Logical CPU:     0
Error Code:      0x02000209
Trap Number:     133


Binary Images:
       0x100b75000 -        0x100b75ff3 +org.python.python3 (3.7.3 - 3.7.3) <5794D7D4-29C2-30D5-9E7A-001132AB5F8F> /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python
       0x100b80000 -        0x100d7bfff +org.python.python3 (3.7.3 - 3.7.3) <7A606CC1-DA05-3C34-BC5C-256C22343EDC> /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/Python3
       0x107af3000 -        0x107b8193f  dyld (732.8) <42C11B81-6928-369F-B03E-D57355572700> /usr/lib/dyld
    0x7fff2eed1000 -     0x7fff2f350ff7  com.apple.CoreFoundation (6.9 - 1671.15) <B888CEC3-88BC-3D1F-BDD7-A664AB66F0BA> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
    0x7fff62bd4000 -     0x7fff62bd6fff  libDiagnosticMessagesClient.dylib (112) <418D550B-C748-3D55-A6D5-0398E032F9F3> /usr/lib/libDiagnosticMessagesClient.dylib
    0x7fff6309b000 -     0x7fff6309cff3  libSystem.B.dylib (1281) <66742D2E-591A-32B2-8E92-4A54BEE843F6> /usr/lib/libSystem.B.dylib
    0x7fff63410000 -     0x7fff63463fff  libc++.1.dylib (800.6) <328FB687-2363-38B1-AC11-11736925C775> /usr/lib/libc++.1.dylib
    0x7fff63464000 -     0x7fff63478fff  libc++abi.dylib (800.7) <02753D3D-91C6-3670-8B5D-EBE040B516FC> /usr/lib/libc++abi.dylib
    0x7fff63ecf000 -     0x7fff63ed1ff7  libfakelink.dylib (149) <FC5712CB-2188-3DAD-8DD4-CC3ECCA3F9A8> /usr/lib/libfakelink.dylib
    0x7fff63fdc000 -     0x7fff64234ff7  libicucore.A.dylib (64232.0.1) <88E47471-605C-3C86-871B-5D2F4628A936> /usr/lib/libicucore.A.dylib
    0x7fff64f56000 -     0x7fff64f87fd6  libobjc.A.dylib (779.1) <D681D734-2739-3D18-9DCB-9ECDA9D492BC> /usr/lib/libobjc.A.dylib
    0x7fff6561f000 -     0x7fff65631fff  libz.1.dylib (76) <3FC3FC3E-ABF3-3167-9078-B54C952608B4> /usr/lib/libz.1.dylib
    0x7fff66096000 -     0x7fff6609bff7  libcache.dylib (83) <8EC69760-6DAA-3068-9372-F1D554C548E5> /usr/lib/system/libcache.dylib
    0x7fff6609c000 -     0x7fff660a7ff7  libcommonCrypto.dylib (60165) <698BE754-137D-361D-B826-57B8DD969E4A> /usr/lib/system/libcommonCrypto.dylib
    0x7fff660a8000 -     0x7fff660affff  libcompiler_rt.dylib (101.2) <0BE7F119-C9C2-3612-A3F4-2336D4444476> /usr/lib/system/libcompiler_rt.dylib
    0x7fff660b0000 -     0x7fff660b9ff7  libcopyfile.dylib (166) <B5E73B1C-0BCF-33FE-80A1-D9E3BA3033D4> /usr/lib/system/libcopyfile.dylib
    0x7fff660ba000 -     0x7fff66151fc3  libcorecrypto.dylib (866.0.10) <58344B13-CD10-3697-A516-6F5B06DD1EB7> /usr/lib/system/libcorecrypto.dylib
    0x7fff66268000 -     0x7fff662a9ff0  libdispatch.dylib (1173.0.3) <F4260D89-F67D-30CB-AF61-7ED25CB687DB> /usr/lib/system/libdispatch.dylib
    0x7fff662aa000 -     0x7fff662dffff  libdyld.dylib (732.8) <98960E27-A08B-36DA-A5CB-8538B2D6757E> /usr/lib/system/libdyld.dylib
    0x7fff662e0000 -     0x7fff662e0ffb  libkeymgr.dylib (30) <682B41BC-BDC2-38D5-8820-87099606FA12> /usr/lib/system/libkeymgr.dylib
    0x7fff662ee000 -     0x7fff662eeff7  liblaunch.dylib (1738.11.1) <7FE11F0D-65BC-3726-B1DD-E84F329193E0> /usr/lib/system/liblaunch.dylib
    0x7fff662ef000 -     0x7fff662f4ff7  libmacho.dylib (949.0.1) <163DFE06-2FAD-3CBC-80EF-C38EED6AEA52> /usr/lib/system/libmacho.dylib
    0x7fff662f5000 -     0x7fff662f7ff3  libquarantine.dylib (110.0.4) <C8F39330-8CB5-30B0-8564-96453DCEFAD7> /usr/lib/system/libquarantine.dylib
    0x7fff662f8000 -     0x7fff662f9ff7  libremovefile.dylib (48) <FB280185-B5ED-3F08-B08A-A378865C1398> /usr/lib/system/libremovefile.dylib
    0x7fff662fa000 -     0x7fff66311fff  libsystem_asl.dylib (377.0.1) <30CE9DAF-B1FA-3510-832B-F1CE19933ED7> /usr/lib/system/libsystem_asl.dylib
    0x7fff66312000 -     0x7fff66312fff  libsystem_blocks.dylib (74) <E0B8C825-E62B-357E-A2A0-13776F0A0F8C> /usr/lib/system/libsystem_blocks.dylib
    0x7fff66313000 -     0x7fff6639aff7  libsystem_c.dylib (1353.11.2) <2A5BFAFE-8214-3B35-AD46-C07A1A8B8941> /usr/lib/system/libsystem_c.dylib
    0x7fff6639b000 -     0x7fff6639efff  libsystem_configuration.dylib (1061.0.2) <56174463-22ED-337F-B0D4-94995DCDB9B7> /usr/lib/system/libsystem_configuration.dylib
    0x7fff6639f000 -     0x7fff663a2ff7  libsystem_coreservices.dylib (114) <01695913-028E-3AE1-8D4E-2B2769109811> /usr/lib/system/libsystem_coreservices.dylib
    0x7fff663a3000 -     0x7fff663aafff  libsystem_darwin.dylib (1353.11.2) <4CE52C63-87AA-3C6D-899F-8C983E5FC061> /usr/lib/system/libsystem_darwin.dylib
    0x7fff663ab000 -     0x7fff663b2ffb  libsystem_dnssd.dylib (1096.0.2) <DC7381E8-F09F-3441-8267-9B8F50A0EBA9> /usr/lib/system/libsystem_dnssd.dylib
    0x7fff663b3000 -     0x7fff663b4ffb  libsystem_featureflags.dylib (17) <DBCA4AA2-CA05-38D5-AB4B-BE0F3E09BB8B> /usr/lib/system/libsystem_featureflags.dylib
    0x7fff663b5000 -     0x7fff66402ff7  libsystem_info.dylib (538) <9F9D6368-A11E-32F1-9BB5-C153F42EFED8> /usr/lib/system/libsystem_info.dylib
    0x7fff66403000 -     0x7fff6642ffff  libsystem_kernel.dylib (6153.11.26) <4CE9D54A-A975-348E-B878-EE74EDFC956B> /usr/lib/system/libsystem_kernel.dylib
    0x7fff66430000 -     0x7fff6647bfe7  libsystem_m.dylib (3178) <FE5E5748-6ACA-3AD2-AF9D-557EB9A3BD7E> /usr/lib/system/libsystem_m.dylib
    0x7fff6647c000 -     0x7fff664a3fff  libsystem_malloc.dylib (283) <02925539-3CBA-39EB-BA6B-9A936CFA0BF8> /usr/lib/system/libsystem_malloc.dylib
    0x7fff664a4000 -     0x7fff664b1ff3  libsystem_networkextension.dylib (1095.11.9) <8B5EE189-E3D1-31FD-878F-50286B6E7047> /usr/lib/system/libsystem_networkextension.dylib
    0x7fff664b2000 -     0x7fff664bbfff  libsystem_notify.dylib (241) <89381127-2A07-3F07-B865-358FACCF9102> /usr/lib/system/libsystem_notify.dylib
    0x7fff664bc000 -     0x7fff664c5fe7  libsystem_platform.dylib (220) <90E508E4-46D8-33FF-8552-DDAA079977A0> /usr/lib/system/libsystem_platform.dylib
    0x7fff664c6000 -     0x7fff664d0fff  libsystem_pthread.dylib (416.11.1) <2EA6F95F-F264-30B6-8AF2-24197B5AED84> /usr/lib/system/libsystem_pthread.dylib
    0x7fff664d1000 -     0x7fff664d5ffb  libsystem_sandbox.dylib (1217.11.16) <51129CBB-BC46-37F1-A1B5-ECFA9530704D> /usr/lib/system/libsystem_sandbox.dylib
    0x7fff664d6000 -     0x7fff664d8fff  libsystem_secinit.dylib (62.0.4) <A48D9AF3-75F2-3331-A0C8-0A23771F4AC7> /usr/lib/system/libsystem_secinit.dylib
    0x7fff664d9000 -     0x7fff664e0ffb  libsystem_symptoms.dylib (1238.0.2) <08E8CF75-5F77-3475-A48E-A01CBDF09173> /usr/lib/system/libsystem_symptoms.dylib
    0x7fff664e1000 -     0x7fff664f7ff2  libsystem_trace.dylib (1147.0.3) <5836645E-9862-326D-AB3B-A19E76BE29B9> /usr/lib/system/libsystem_trace.dylib
    0x7fff664f9000 -     0x7fff664feffb  libunwind.dylib (35.4) <F5AE1D43-7C5F-3BA2-94D3-E769F82C0F61> /usr/lib/system/libunwind.dylib
    0x7fff664ff000 -     0x7fff66533ff6  libxpc.dylib (1738.11.1) <2E9076CD-6C0E-38B6-8403-B2DDCE125FBF> /usr/lib/system/libxpc.dylib

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 15256
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=362.7M resident=0K(0%) swapped_out_or_unallocated=362.7M(100%)
Writable regions: Total=53.6M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=53.6M(100%)
 
                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        1 
MALLOC                            36.3M       16 
MALLOC guard page                   16K        3 
STACK GUARD                          4K        1 
Stack                             16.0M        2 
VM_ALLOCATE                       1024K        4 
__DATA                            2577K       48 
__DATA_CONST                        20K        1 
__LINKEDIT                       349.1M        4 
__OBJC_RO                         31.8M        1 
__OBJC_RW                         1764K        2 
__TEXT                            13.6M       46 
__UNICODE                          564K        1 
shared memory                        8K        2 
===========                     =======  ======= 
TOTAL                            452.6M      132 

Model: MacBookPro10,2, BootROM 281.0.0.0.0, 2 processors, Dual-Core Intel Core i5, 2.6 GHz, 8 GB, SMC 2.6f59
Graphics: kHW_IntelHD4000Item, Intel HD Graphics 4000, spdisplays_builtin
Memory Module: BANK 0/DIMM0, 4 GB, DDR3, 1600 MHz, 0x802C, 0x384B54463531323634485A2D314736453120
Memory Module: BANK 1/DIMM0, 4 GB, DDR3, 1600 MHz, 0x802C, 0x384B54463531323634485A2D314736453120
AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x10F), Broadcom BCM43xx 1.0 (7.21.190.33 AirPortDriverBrcm4360-1408)
Bluetooth: Version 7.0.0f8, 3 services, 18 devices, 1 incoming serial ports
Network Service: Wi-Fi 2, AirPort, en0
Serial ATA Device: APPLE SSD SD512E, 500.28 GB
USB Device: USB 2.0 Bus
USB Device: Hub
USB Device: Hub
USB Device: Apple Internal Keyboard / Trackpad
USB Device: BRCM20702 Hub
USB Device: Bluetooth USB Host Controller
USB Device: USB 2.0 Bus
USB Device: Hub
USB Device: FaceTime HD Camera (Built-in)
USB Device: USB 3.0 Bus
USB Device: Microsoft® 2.4GHz Transceiver v9.0
Thunderbolt Bus: MacBook Pro, Apple Inc., 23.4
```


Discussion:
The only error I see when building Python is:
```
In file included from /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/spack-src/Modules/_tkinter.c:48:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/tk.h:86:11: fatal error: 'X11/Xlib.h' file not found
#       include <X11/Xlib.h>
                ^~~~~~~~~~~~
```
but I'm building `python~tk`. I also see this during installation:
```
In file included from /private/var/folders/21/hwq39zyj4g36x6zjfyl5l8080000gn/T/Adam/spack-stage/spack-stage-python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/spack-src/Modules/_tkinter.c:48:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/tk.h:86:11: fatal error: 'X11/Xlib.h' file not found
#       include <X11/Xlib.h>
                ^~~~~~~~~~~~
1 error generated.
...
Listing '/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8'...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'multiprocessing.resource_tracker'
...
Listing '/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/test/sndhdrdata'...Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/lib/python3.7/multiprocessing/spawn.py", line 115, in _main
    self = reduction.pickle.load(from_parent)
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/concurrent/futures/__init__.py", line 8, in <module>
    from concurrent.futures._base import (FIRST_COMPLETED,
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/concurrent/futures/_base.py", line 7, in <module>
    import logging
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/logging/__init__.py", line 28, in <module>
    from string import Template
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/string.py", line 109
    def substitute(self, mapping=_sentinel_dict, /, **kws):
                                                 ^
SyntaxError: invalid syntax
...
Listing '/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/tkinter/test'...
Traceback (most recent call last):
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/compileall.py", line 327, in <module>
    exit_status = int(not main())
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/compileall.py", line 309, in main
    if not compile_dir(dest, maxlevels, args.ddir,
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/compileall.py", line 86, in compile_dir
    results = executor.map(partial(compile_file,
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/concurrent/futures/process.py", line 674, in map
    results = super().map(partial(_process_chunk, fn),
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/concurrent/futures/_base.py", line 600, in map
    fs = [self.submit(fn, *args) for args in zip(*iterables)]
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/concurrent/futures/_base.py", line 600, in <listcomp>
    fs = [self.submit(fn, *args) for args in zip(*iterables)]
  File "/Users/Adam/spack/opt/spack/darwin-catalina-ivybridge/clang-11.0.0-apple/python-3.8.0-vpc6afg6xwxk7pjzrw7plscslxdki6fy/lib/python3.8/concurrent/futures/process.py", line 629, in submit
    raise BrokenProcessPool(self._broken)
concurrent.futures.process.BrokenProcessPool: A child process terminated abruptly, the process pool is not usable anymore
make: [libinstall] Error 1 (ignored)
```
Homebrew and Anaconda haven't updated to Python 3.8 yet, so it's hard to tell if this is a bug in Python or not.
For the record, I don't see any of these build errors with Python 3.7.3. The build script correctly detects that Tk/Tcl are not available and never tries to build Tkinter. I'm beginning to think that this is a Python 3.8 bug. The unit tests also don't pass.

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/python/package.py b/var/spack/repos/builtin/packages/python/package.py
--- a/var/spack/repos/builtin/packages/python/package.py
+++ b/var/spack/repos/builtin/packages/python/package.py
@@ -31,6 +31,7 @@ class Python(AutotoolsPackage):
 
     maintainers = ['adamjstewart']
 
+    version('3.8.2',  sha256='e634a7a74776c2b89516b2e013dda1728c89c8149b9863b8cea21946daf9d561')
     version('3.8.1',  sha256='c7cfa39a43b994621b245e029769e9126caa2a93571cee2e743b213cceac35fb')
     version('3.8.0',  sha256='f1069ad3cae8e7ec467aa98a6565a62a48ef196cb8f1455a245a08db5e1792df')
     version('3.7.6',  sha256='aeee681c235ad336af116f08ab6563361a0c81c537072c1b309d6e4050aa2114', preferred=True)


Test patch: