Issue 1:
I'm using python shared memory from multiprocessing package to share numpy arrays between processes.

There is a caveat, however, that populating such objects from buffer requires duplicate copy operation.

np.frombuffer creates new array and there is no option to populate existing array which is linked with a shared memory buffer.

In result, it is required to run 2 operations: deserializing the buffer content to numpy object and the second to copy elements from that array to the one linked with shared memory.

I would like to propose extending np.frombuffer to take additional argument of existing numpy array that would be populated with the content instead of creating new one.

Reproducing code example:

import numpy as np
from multiprocessing import shared_memory

input = np.ones((1,10,10,10))
b = input.tobytes()

input_shm = shared_memory.SharedMemory(create=True, size=input.nbytes)
write_array = np.ndarray(input.shape, dtype=input.dtype,buffer=input_shm.buf)

content = np.frombuffer(b,dtype=input.dtype).reshape(input.shape) # this is one copy extensive operation

write_array[:] = content[:] # this is duplicate copy to populate shared memory buffer

restored_numpy = np.ndarray(input.shape, dtype=input.dtype,buffer=input_shm.buf)
# reading from shared memory is quick - attaching the memory block

print("restored",restored_numpy)

The desired behaviour would be like:

np.frombuffer(b,dtype=input.dtype,target_ndarray=write_array)
# deserialization and populating the shared memory buffer at one time

Numpy/Python version information:

import sys, numpy; print(numpy.version, sys.version)

1.18.1 3.8.1 (default, Jan 22 2020, 16:31:49)

[GCC 9.2.1 20200111 gcc-9-branch@280154]

Issue 2:
I ran into this, and the solution found on numpy-discussion worked for me, but does

turn up on a search of the issues.

Summary:

Python <3.2 are compiled with VS2008 (SDK v7.0,"sdk for win7 + NET 3.5.1")

Python 3.3 (+?) is compiled with VS2010 (SDK v7.1, "sdk for win7 + NET 4")

It's often recommended that c extensions (such as numpy) be compiled using the same compiler

used to compile the python binaries, to ensure CRT compatability, so using VS2010/SDK 7.1 to compile

numpy for python 3.3 is implied.

VS2010 (msvc10) behaves differently with regards to manifest files then VS2008 (msvc9) but the cpython distutils, which the numpy build process relies on,

have not been altered to allow for this (as of python 3.3.3).

a cpython issue was filed in dec/2012, but has stalled.

The issue manifests as a "broken toolchain" exception when attempting to build

numpy (both 1.7.1 and 1.8.0), which results from the following compiler error message.

Found executable C:\Program Files\Microsoft SDKs\Windows\v7.0A\bin\mt.exe

_configtest.exe.manifest : general error c1010070: Failed to load and
parse the manifest. The system cannot find the file specified.

The fix suggested by Christoph Gohlke on the ml is to edit the file "Lib/distutils/msvc9compiler.py:648", Located under your python 3.3 install directory, and modify the line from:

mfinfo = self.manifest_get_embed_info(target_desc, ld_args)

to

mfinfo = None

I've tried, and it has worked for me. Thanks goes to @cgohlke, the intrepid crusader for windows pydata binaries.

There, that should give pagerank something to chew on.