pydata/pandas — pull request #3297 by y-p (2013-04-09 18:24:51+00:00)
Title: WIP: add .meta attribute to frame and series, which gets de/serialized

#2485

While collecting vbench data recently, it became painfully
obvious how being able to serialize metadata information
along with pickled frames.

This is a rough draft for something I'd like to see happen in 0.12.
It creates yet another version of pickle files, so much testing
needs to be done there.

There's a plan (#686, timeline unclear) for implementing a binary serialization format for pandas, which will need to replicate this functionality.

@jreback, if you have something planned for this, I'm glad to
withdraw this PR, it's just a statement of intent.

The design choices I'm going for right now:
- **no propagation** semantics through operations, the metadata tags are restored
if present, and are guranteed on load, only.
Example use case : store measurements as dataframes with data,
location, etc'. Then, load a mess of them back up and (generally) you may
use the metadata either as column/index labels, or just pick out a subset
based on them, sort based on them, etc'.
- only JSONable data allowed in dicts. Allowing anything else would bring
us back to the pickle problem, might create issues if #686 gets implemented
(difficulty in serializing metadata without using pickl), and is generally
an unknown quantity. JSON can cover a lot of mileage.
letting objects define their own way to be serialized for metadata purposes
is reinventing pickle, so no. (JSONable not yet enforced in the code).
- still need to consider the binary blob case.
- reserved keywords/namespace for pandas' use
- make it minimal, no setdefault() etc, users should roll their own there.
- currently, it's just a dict `.meta` that gets pickled and unpickled with the object.
- The usual I want tab completion/attr access/naming discussion to
happen.
- versioning/schema to be baked in as reserved keywords? the pickle
code taught me that much.

```python
In [8]: df=mkdf(10,5)
...: df.meta
...: df.meta['Im']="persistent"
...: df.save('/tmp/1.pickle')
...: dfrt =pd.load('/tmp/1.pickle')
...:

In [9]: dfrt.meta
Out[9]: {'Im': 'persistent'}
```