freeman-lab/thunder — pull request #43 by industrial-sloth (2014-10-24 01:13:30+00:00)
Title: 0.4.x: indexing and bug fixes

Big set of fixes here.

Most of the code changes here go toward ensuring that array shape, series indicies, and so on are handled consistently across data types and loading methods. Binary stack dimensions should be specified on loading as (x, y, z) (assuming that z is the most slowly-varying dimension in the data on disk, as it typically will be), and will come out after a pack() as an array of the same shape.

Other bug fixes:
* a data corruption bug coming from a float16() cast in pack() (works fine, unless data value uses more than 11 bits...)
* a bug in tif loading where certain files would lead to an exception from trying to pop() an empty list
* a bug in tif loading where if multiple blocks were specified within a single tif page, in some cases the full page would not be read
* a bug in reading from s3 where in some cases an S3 exception would be thrown from trying to read past the end of a file
* several inconsistencies and general flakiness in handling path strings, reported by Joseph Naegele - thanks! (path handling probably still isn't perfect, but it's improved, at least.)
* possible fix for possible / likely memory leak when reading multipage tifs with shuffle=False. Still needs to be verified by testing at scale.

New functionality:
* add optional 'datatype' parameter on thunder context load methods, allowing binary image data of types other than int16 to be loaded. (Oops. Thanks again Joseph Naegele.)