Package: kson
KSON: Keystripped Schemafied Object Notation============================================KSON is a simple data interchange format based on JSON. Its serializedrepresentation doesn't contain the redundant keys of typical jsondocuments and more compact representations of values are possible throughan extensible encoding/decoding mechanism.KSON uses a simple schema format to describe arbitrarrily nested objectsand encoded types. The minified and gzipped javascript library is < 1K.Usage in javascript-------------------// Load schema definitionsvar movie_schemas = [{"id": "role","fields": ["name", "character"],"meta": [0, 0]},{"id": "movie","fields": ["title", "year", "rating", "cover", "actors"],"meta": [0, 0, 0, "prefix:http\://movies.db/covers/|suffix.jpg", "[]role"]}];KSON.addSchema(movie_schemas);var movies = [{"title": "Forrest Gump","year": 1994,"rating": 8.7,"cover": "http://movies.db/covers/8.jpg","actors": [{"name": "Tom Hanks", "character": "Forest Gump"},{"name": "Robin Wright", "character": "Jenny Curran"},{"name": "Gary Sinise", "character": "Lieutenant Dan Taylor"}]},{"title": "Toy Story",...}...];// pretty printing is not part of the libraryconsole.log(KSON.stringify(movies, "[]movie"));["[]movie","Forest Gump",1994,8.7,"8",[["Tom Hanks", "Forest Gump"],["Robin Wright", "Jenny Curran"],["Gary Sinise", "Lieutenant Dan Taylor"]],"Toy Story",...]// Of course the "movie" and "role" schema definitions may also be// serialized to KSON using the "schema" schema:var schema_data = KSON.stringify(movie_schemas, "[]schema")console.log(schema_data);["[]schema","role",["name","character"],[0, 0],"movie",["title","year","rating","cover","actors"],[0,0,0,"prefix:http\://movies.db/covers/|suffix:.jpg","[]role"]]// Likewise schemas can be initialized with a kson documentKSON.addSchema(schema_data);FAQ---Q: Why should I use this instead of JSON/MessagePack/Thrift/...?A: Unless you are targeting the browser you probably shouldn't. If youare, then the size and speed of the javascript libraries for theseformats may make KSON a favorable option.Q: Is KSON faster/smaller/better than JSON?A: Tests using node.js indicate that parsing is marginally slower. Thisshould be outweighed by faster transmission speed of signifigantly lessdata. See the node.js benchmarks for more info.Q: Is this a drop in replacement for JSON?A: No, you will need to write schema definitions for your data. Automaticschema detection based on example data may help in getting started. Alsosee "restrictions" in the Specification.Q: Why yet another a schema format for JSON?A: Existing JSON schema formats include extrenious information to thepurpose of KSON and would require larger javascript library forparsing and serialization.Q: Is there language support for php/ruby/java/c...?A: Currently javascript and python are supported. Porting to otherlanguages with existing JSON support should be fairly easy and patchesare very welcome.Schema Specification--------------------A KSON schema consists of three fields:id              *string* identifierfields          *array* of field namesmeta            *array* signifying the types associated withcorresponding elements in the *fields* array.A schema only defines one object. Nesting is accomplished by referencingsubschemas via their id in the meta array.An element of *meta* specifies the content of the element of *fields* atthe same index. The lengths of the *fields* and *meta* arrays must be equal.An element of *meta* must be one of the following:0                field contains a plain value of type*string*, *boolean*, *number*, *null*."[]"             *array* of plain values"schema-id"      *string* reference the schema of a nested object"[]schema-id""codec-id""[]codec-id"The top level elements of a KSON document must be either an *array* ofobjects or an *object*. A plain value or an *array* of plain valuescannot be serialized.Codecs------Decoders are executed in the order of their definition in the meta field.The decoder is passed the value to decode as the first argument and theintermediate data object as the second. The intermediate object will havethe values of all previous fields decoded and applied.Included codecs:enum:a:b:c:d   "a" -> 0, "b" -> 1, ...prefix:egg     "eggbacon" -> "bacon"suffix:bacon   "eggsausagebacon" -> "eggsausage"bool           true -> 1, false -> 0date           new Date() -> 1364938727390int36          1364938727390 -> "hf1lcnka"Codec chainingprefix:egg|suffix:bacon    "eggsausagebacon" -> "sausage"date|int36    new Date() -> "hf1lcnka"Custom codecs are registered using factory functions in the following manner.KSON.coders["my_coder_id"] = function(args) {// args is a string array from a schemas meta field// enum:a:b:c:d -> args == ["a", "b", "c", "d"]return function (val, encode) {// The actual encoding/decoding functionreturn encode ? encode_op(val) : decode_op(val);}}Schema Validation-----------------The python library validates schemas and codec definitions, the javascriptlibrary does not. Invalid schema definition loaded with the javascriptlibrary may parse or stringify bad data without any throwing any errors.Common errors include:- Loading a schema before the custom codecs it uses have been registered.- Loading a schema before the subschemas on which it depends have beenloaded. Note that schemas are loaded in the order they appear in a file,so dependent schmas schould come after those they depend uppon.- Circularly dependent schmas (including schemas which depend onthemselves) are possible by using a stub schema, which only specifiesan id and no fields. This can be replaced by the propper schema withthe same id, after its dependencies have been loaded.- Loading a json encoded schema. The argument of the KSON.addSchemafunction may be either a KSON encoded schema or schemas, an alreadydecoded schema or an array of schemas. It is however fairly simple toload json encoded schemas: KSON.addSchema(JSON.parse(raw_json_schemas));The python library will warn about these and other issues, so use it toInstallation------------$ pip install ksonAutomatic schema detection:$ kson --auto-detect data.json --id schema_idDevelopment-----------$ git clone git@bitbucket.org:mbarkhau/kson.git$ npm install uglify-js -g$ npm install benchmark microtime$ pip install py.test$ make test$ make bench$ make minify