kevinweil/elephant-bird — pull request #176 by panisson (2012-04-04 10:56:56+00:00)
Title: Load nested JSON structures

Hello, this is another proposal for the implementation of loading nested JSON structures using JsonLoader.
There are other merge proposals, but as we were in desperate need of an updated implementation of such feature, I implemented it and it worked well using pig-0.9.2. As it was very useful for us, we thought that it would be useful for other elephant-bird users as well, and we decided to submit a pull request.
This implementation loads JSON objects and arrays. Objects are loaded as nested Maps and arrays are loaded as Bags. This approach makes easy to access nested structures and expand array data using FLATTEN (see example on TestJsonLoader.testNestedLoad).
This behavior can be easily disabled by using a pig property, making it backwards compatible. The default behavior is to load nested structures.
Another feature that was implemented (inspired by pull request #128) is the use of an optional constructor to specify the JSON keys that should be loaded. If the constructor is not used, all JSON structures are loaded, otherwise only the JSON keys specified by the constructor are loaded, speeding up the load and minimizing memory usage.