Writing a little wrapper around file.read that provides fgetc, unget, read,
and 1 character of buffering would take like... 15 lines of code.


On Mon, Mar 17, 2014 at 7:56 PM, Julian Taylor <notifications@github.com>wrote:

> In numpy/core/src/multiarray/multiarraymodule.c:
>
> > @@ -2001,10 +2019,48 @@
> >      }
> >      fp = npy_PyFile_Dup2(file, "rb", &orig_pos);
> >      if (fp == NULL) {
> > -        PyErr_SetString(PyExc_IOError,
> > -                "first argument must be an open file");
> > -        Py_DECREF(file);
> > -        return NULL;
> > +        /* Try handling file-like objects that are not file */
> > +        if (PyArray_IsFileLike(file)) {
> > +            /* create tempfile and fill it with repetitions of obj.read() */
> > +            fp = tmpfile();
> > +
> > +            while (1) {
> > +                buff = PyObject_CallMethod(file, "read", NULL);
>
> for binary files we have no issue, for some string stuff we need fgetc and
> unget, but I think in total having custom buffering for that would have
> been a lot faster than all the hacks we have now.
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/numpy/numpy/pull/4505/files#r10675543>
> .
>



-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org