I have to ask -- how hard would it be to teach fromfile to read from
file-like objects directly?


All we need is "read", right? It's not like the boilerplate to call a
"read" method is *that* more complicated than fread().


On Mon, Mar 17, 2014 at 7:52 PM, Julian Taylor <notifications@github.com>wrote:

> In numpy/core/src/multiarray/multiarraymodule.c:
>
> > @@ -2001,10 +2019,48 @@
> >      }
> >      fp = npy_PyFile_Dup2(file, "rb", &orig_pos);
> >      if (fp == NULL) {
> > -        PyErr_SetString(PyExc_IOError,
> > -                "first argument must be an open file");
> > -        Py_DECREF(file);
> > -        return NULL;
> > +        /* Try handling file-like objects that are not file */
> > +        if (PyArray_IsFileLike(file)) {
> > +            /* create tempfile and fill it with repetitions of obj.read() */
> > +            fp = tmpfile();
> > +
> > +            while (1) {
> > +                buff = PyObject_CallMethod(file, "read", NULL);
>
> I reading in chunks would be save memory, even better would be reading
> into a pipe from a thread but portability issues ...
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/numpy/numpy/pull/4505/files#r10675358>
> .
>



-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org