Repository: conda/conda-build

Issue:
CPU_COUNT reliability in MacOSX
Hello!
After a talk with @ccordoba12 in [conda-recipes/pr427](https://github.com/conda/conda-recipes/pull/427), it seems that CPU_COUNT is not reliable in MacOSX and he suggests to use "sysctl -n hw.ncpu".

Have you encountered this bug ?
I see that in python3.5, multiprocessing.cpu_count uses os.cpu_count and this builtin command uses also sysctl : posixmodule.c, line 11220:

``` c
#elif defined(__DragonFly__) || \
      defined(__OpenBSD__)   || \
      defined(__FreeBSD__)   || \
      defined(__NetBSD__)    || \
      defined(__APPLE__)
    int mib[2];
    size_t len = sizeof(ncpu);
    mib[0] = CTL_HW;
    mib[1] = HW_NCPU;
    if (sysctl(mib, 2, &ncpu, &len, NULL, 0) != 0)
        ncpu = 0;
```

To fix it, I can create a patch that add an OSX case in in [conda-build/environ.py:109](https://github.com/conda/conda-build/blob/master/conda_build/environ.py#L109) with something like:

``` python
import subprocess
import sys

if sys.platform == 'darwin':
    out, err = subprocess.Popen('sysctl -n hw.ncpu', shell=True, stdout=subprocess.PIPE).communicate()
    d['CPU_COUNT'] = str(out).strip()
```

Do you agree on this approach ?
Thank you very much

CPU_COUNT reliability in MacOSX
Hello!
After a talk with @ccordoba12 in [conda-recipes/pr427](https://github.com/conda/conda-recipes/pull/427), it seems that CPU_COUNT is not reliable in MacOSX and he suggests to use "sysctl -n hw.ncpu".

Have you encountered this bug ?
I see that in python3.5, multiprocessing.cpu_count uses os.cpu_count and this builtin command uses also sysctl : posixmodule.c, line 11220:

``` c
#elif defined(__DragonFly__) || \
      defined(__OpenBSD__)   || \
      defined(__FreeBSD__)   || \
      defined(__NetBSD__)    || \
      defined(__APPLE__)
    int mib[2];
    size_t len = sizeof(ncpu);
    mib[0] = CTL_HW;
    mib[1] = HW_NCPU;
    if (sysctl(mib, 2, &ncpu, &len, NULL, 0) != 0)
        ncpu = 0;
```

To fix it, I can create a patch that add an OSX case in in [conda-build/environ.py:109](https://github.com/conda/conda-build/blob/master/conda_build/environ.py#L109) with something like:

``` python
import subprocess
import sys

if sys.platform == 'darwin':
    out, err = subprocess.Popen('sysctl -n hw.ncpu', shell=True, stdout=subprocess.PIPE).communicate()
    d['CPU_COUNT'] = str(out).strip()
```

Do you agree on this approach ?
Thank you very much



Discussion:
An enlightening issue on Python issue tracker.
https://bugs.python.org/issue17444

According to this issue, it looks like it might be preferable to use `hw.logicalcpu` instead of `hw.ncpu`

@groutr and @msarahan, what do you think about this one?

I am in favor of it. I'd like some answer to @groutr suggestion above before it is merged, though.

@gbaty seems like everyone is in agreement here, as long as you include @groutr's suggestion.  If you want to open a PR we'll get it approved.

An enlightening issue on Python issue tracker.
https://bugs.python.org/issue17444

According to this issue, it looks like it might be preferable to use `hw.logicalcpu` instead of `hw.ncpu`

@groutr and @msarahan, what do you think about this one?

I am in favor of it. I'd like some answer to @groutr suggestion above before it is merged, though.

@gbaty seems like everyone is in agreement here, as long as you include @groutr's suggestion.  If you want to open a PR we'll get it approved.


Resolving patch:
diff --git a/conda_build/environ.py b/conda_build/environ.py
--- a/conda_build/environ.py
+++ b/conda_build/environ.py
@@ -106,10 +106,16 @@ def get_dict(m=None, prefix=None):
                 value = '<UNDEFINED>'
             d[var_name] = value
 
-    try:
-        d['CPU_COUNT'] = str(multiprocessing.cpu_count())
-    except NotImplementedError:
-        d['CPU_COUNT'] = "1"
+    if sys.platform == "darwin":
+        # multiprocessing.cpu_count() is not reliable on OSX
+        # See issue #645 on github.com/conda/conda-build
+        out, err = subprocess.Popen('sysctl -n hw.logicalcpu', shell=True, stdout=subprocess.PIPE).communicate()
+        d['CPU_COUNT'] = out.decode('utf-8').strip()
+    else:
+        try:
+            d['CPU_COUNT'] = str(multiprocessing.cpu_count())
+        except NotImplementedError:
+            d['CPU_COUNT'] = "1"
 
     d.update(**get_git_build_info(d['SRC_DIR']))
     d['PATH'] = dict(os.environ)['PATH']


Test patch: