freeman-lab/thunder — pull request #44 by industrial-sloth (2014-10-28 21:46:05+00:00)
Title: Data casting methods

This PR adds an `astype()` method on Data objects, which basically just calls numpy's astype() method in parallel on the underlying RDD values.
It also moves the `dtype` (numpy data type) attribute up to the Data superclass, where previously it had just been on Images, and adds `dtype` as a convenience parameter on Series.pack and the various stats methods (mean, variance, etc) on Data.
In addition to the standard numpy dtype specifiers, Data.astype() can also accept a dtype of 'smallfloat', which will evaluate to the smallest floating point representation to which the current dtype can be safely cast. For ints and uints, this will be a float that is larger than the int type (int16 goes to float32, etc). The extra bits used to represent the exponent mean that in general, one can't cast an integer safely to a float of the same size - of course, if you know that your uint16s have no values larger that 2047, you're welcome to cast them explicitly to float16s.
The Data stats methods use 'smallfloat' as their default parameter value, since we assume that usually if you're calculating a mean, and definitely if you're calculating a variance, you want to get floats back - but since there's a good change that these values are going to get pushed across a network connection, we don't want to use more space than we have to.