apache/incubator-flink — pull request #74 by tillrohrmann (2014-07-17 17:14:53+00:00)
Title: [FLINK-610] Added KryoSerializer

I added the KryoSerializer and replaced the AvroSerializer as the standard generic type serializer. Due to the way Flink's serialization works, we cannot exploit Kryo's Inputs which buffer data for serialization. Instead, we have to read byte after byte if we do not know the length of the underlying data. This has consequences for variable length encodings. Kryo will fall back to a slow deserialization of these types, since it will read only one byte at a time. I do not know the implications of this on Kryo's performance. Therefore, I did not remove Avro completely yet. Furthermore, Kryo should be tested on the cluster and if possible its performance should be compared to Avro.

We could mitigate the problem if we don't use variable length encodings for types such as ```int```s and ```long```s. This, however, comes at the price of larger serialized objects. For UTF8 strings, the problem would persist.