Project: datamanagement
Issue DM-4734: afw fails to build on a machine with many cores
The afw package does not build reliably (if at all) on a linux box at UW ("magneto", which has 32 cores and 128 Gb of RAM). The failure is that some unit tests fail with the following error: {code} OpenBLAS: pthread_creat error in blas_thread_init function. Error code:11 {code} For the record, /usr/include/bits/local_lim.h contains this: {code} /* The number of threads per process. */ #define _POSIX_THREAD_THREADS_MAX	64 /* We have no predefined limit on the number of threads. */ #undef PTHREAD_THREADS_MAX {code} It appears that the build system is trying to use too many threads when building afw, which presumably means it is trying to use too many cores. According to [~mjuric] the package responsible for this is {{eupspkg}}, and it tries to use all available cores. A workaround suggested by [~mjuric] is to set environment variable {{EUPSPKG_NJOBS}} to the max number of cores wanted. However, I suggest we fix our build system so that setting this variable is unnecessary. I suggest we hard-code an upper limit for now, though fancier logic is certainly possible. A related request is to document the environment variables that control our build system. I searched for {{NJOBS}} on confluence and found nothing.
Story points: 1