opened
mhowlett
username_0
this PR takes a look at what happens if `ISerializer/IDeserializer` are replaced by delegates. 

Notes:
- we can't express the delegates as a `Func`.. since `ReadOnlySpan<T>` can't be a generic parameter.
- the serializers are very specific for use with the kafka clients (regardless of their form) - they need a topic parameter and the serializer is likely going to have a `IMemoryOwner<byte>` parameter, which is a general concept but that's pretty advanced, limiting general usefulness.
- there is structural benefit from re-organizing things so that serdes are not `IDisposable`s - the API is easier to 'get' - the user doesn't need to wonder about transfer of ownership.
- coupling is reduced substantially if the current configuration mechanism is removed. the way things work would be more straightforward (to the user). the disadvantage is that the way things work now does add value - it allows a single config dictionary to work for every thing AND provide fail fast functionality if something is wrong. However, the same functionality can be achieved with a few `.Select` statements prior to passing a config dictionary to the various bits.
- There is perf. advantage to having the consumer run the deserializer as part of the consume call (can be done before librdkafka memory is freed). for this reason, it wouldn't be good to decouple deserializer and consumer completely.
- On the producer, avro serializers need to be topic aware. having the serialization performed by the producer means the topic doesn't need to be specified multiple times. a reason not to decouple serializer/producer completely.
- I've added UTF8 (string) and `byte[]` to the (de)serializer types that don't need to be explicitly specified. it could be argued that having strings implicitly default to UTF8 is not a good feature. i could remove this, but I think i'm in favor of it. I don't like the look of `Serializers.UTF8` on initialization, and i don't think defaulting to UTF8 will cause problems.
ISerializer/IDeserializer -> delegates [WIP]
issue
created
mhowlett
username_0
more notes:

- I played with removing `topic` from the (de)serializer delegates, since the only place this is used is in the avro serializer and the pull of simplicity is strong - perhaps it's better with going with simplicity, and anything that needs more can build on top of the basics. However, any API I came up with was worse, in my view, from a users perspective, and central registration of schemas is an important architectural pattern we believe many people should use, so we want this usage to be straightforward.

- something else we could do is go the other way and give everything passed into the produce method to the serializer. @vchekan suggests header values would be useful.

- An advantage of typed producers/consumers is type safety.

- Another advantage of typed producers/consumers is terseness. Although straightforward to serialize / deserialize to/from bytes, in practice I find this is more characters of typing.

- The basic auth PR I have open follows other clients and uses `SASL_INHERIT` as a potential source. Implicitly, this assumes a single config is used for everything. This PR breaks that way of thinking.
comment