saltstack/salt — pull request #10625 by johnnoone (2014-02-20 18:35:26+00:00)
Title: [PROPOSAL] serializers & aggregation

Hello folks,

I want to propose these two new utils libraries that lay the foundations of saltstack/salt#3991, saltstack/salt#2466 and other .sls files misbehaviors.

### salt.utils.aggregation

salt.utils.aggregation is a smart merging tool for Python objects.

for example:

```python
from salt.utils.aggregation import aggregate, Scalar

# explicit aggregation
a = Scalar('first')
b = Scalar('second')
assert aggregate(a, b) == ["first", "second"]

# introspection on nested values
a = {
"foo": {
"bar": Scalar('first'),
"baz": "should not be displayed"
}
}
b = {
"foo": {
"bar": Scalar('second')
}
}
assert aggregate(a, b) == {"foo": {"bar": ["first", "second"]}}

```

### salt.utils.serializers

salt.utils.serializers is a start for cleanup data serialization/deserialization issues.

Currently is implements **yaml**, **json**, **msgpack** and a fresh **sls** data structures format with their optims & fallback.

The most notable feature between these codecs is that they implement the same interfaces. For example:

```python
from salt.utils.serializers import json, yaml, msgpack, sls

for codec in (json, yaml, msgpack, sls):
obj = codec.decode(src)
codec.encode(obj)
```

I've also added a new codec named **sls** based yaml, which able to handle all the salt magic:

- defacto OrderedDict
- yaml nice printing (for templates)
- data aggregation with ``!aggregation`` yaml tag, based on the salt.utils.aggregation.

for example:

```python
from salt.utils.serializers import sls

src = "{foo: bar, baz: qux}"
obj = sls.decode(src)

# defacto OrderedDict
assert isinstance(obj, OrderedDict)

# yaml nice printing
assert obj.__str__() == "{foo: bar, baz: qux}"
assert obj["foo"].__str__() == "\"bar\""
```

and instructed aggregation within the ``!aggregation`` tag:

```yaml
# pillar.sls
foo: !aggregate first
foo: !aggregate second
bar: !aggregate {first: foo}
bar: !aggregate {second: bar}
baz: !aggregate 42
```

is roughly equivalent to

```yaml
foo: [first, second]
bar: {first: foo, second: bar}
baz: [42]
```

more examples and use cases are present in the unit tests.