BUG: TimeSeries compat from < 0.13
State: closed

fixes a pickle compat issue for < 0.13 when presented with a TimeSeries (was an oversight on the initial PR).

additional tests as well


Labels:

Comments:
@jtratner @cpcloud 

can you guys have a look at this under py3. There are some 'errors' when I try to read a py2.7 in the compatibility mode (e.g. when I have to 'fake' a ndarray sub-class to make it work). Works fine for py2 reading py2 and py3 reading py3. But get a very strange error when reading py2 with py3.

It just prints the output for now.

Try this (you can also drop into the debugger at the print statement as well)

```
[goat-jreback-~/pandas] nosetests-3.2 build/lib.linux-x86_64-3.2/pandas/io/tests/test_pickle.py -s

(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x456b8c0>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.10.1/AMD64_windows_2.7.3.pickle : initializing object must be a string
(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x456b908>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.10.1/x86_64_linux_2.7.3.pickle : initializing object must be a string
./mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.11.0/test.pickle : 'numpy.int64' object has no attribute 'items'
(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x457e3b0>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.11.0/x86_64_linux_2.7.3.pickle : initializing object must be a string
.(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x45c4ef0>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.12.0/0.12.0_i686_linux_2.7.3.pickle : initializing object must be a string
(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x45c4f80>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.12.0/0.12.0_x86_64_linux_2.7.3.pickle : initializing object must be a string
(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x45c4b48>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.12.0/x86_64_linux_2.7.3.pickle : initializing object must be a string
..
----------------------------------------------------------------------
Ran 4 tests in 0.125s
```

I'll definitely take a look...

@jreback the test cases are setup now such that they never fail :P (you need to add a `raise` after the print exception part or something...

that was on purpose, so u could look

any idea about the unpickle error?

ah okay - no I have no idea, I was trying to grep/grin around in the Python source to find what raises the error and I couldn't find anything with 'initializing object'. Any ideas on where it's raised?

yep that's the problem
it's unpicking a np.dtype('float'64) object 
but for some reason it has a weird string encoding - cant see any reason for that

maybe you're overriding some part that sets an encoding internally in the pickle module?  Also, have you tried unpickling but removing pickle_compat? (do you still see the 'initialize string' error)?

I first try regular unpickle (this is the `try_read` bizness in io/pickle.py, then fallback to the compat decoding, then try encoding in the same way.

This 'fails' on the final try, e.g. using pickle_compat with the encoding. AFAIK I am doing it correctly. for example. in 3.3 it correctly unpickles a 3.3 from 0.11.

Its just the py2 with pickle_compat that is failing with something I am not even touching.....

Whee... Did 3.3 unpickle a Python 2 Series pickle before you did the Series
refactor?

Found it. 

```
(Pdb) p args[1]
'\x00\x00\x00\x00\x00\x00ø\x7f'
(Pdb) p func(args[0],args[1].encode(self.encoding))
nan
```

I think this might be a numpy/python bug. `args[0]` here is a `np.dtype('float64')` object and the value (`args[1]`) is a string. But the dtype is expecting bytes. So I just catch the error and reencode the strings and it works. 

really odd, like the bytes were magically converted to a string w/o properly decoding. (and the result is a `nan`!)