kevin-montrose/Jil — pull request #70 by manofstick (2014-09-19 10:13:01+00:00)
Title: Deserialize with static sized buffer

OK; well this (as well as 'Unsafe write encoded string 3') is an attempt to break the impasse of getting these StringBuilder alternatives across the line.

Given the information you have provided re: your usage, I am becoming increasingly convinced that it is only the user of the library that can provide hints about how much memory the process can consume. Obviously something needs to grow to handle the biggest string, but I think that should remain the job of StringBuilder, which presumably has been created with all best practices of how to act nicely with the garbage collector (i.e. some sort of [rope](http://en.wikipedia.org/wiki/Rope_(data_structure)) which doesn't exceed the "large object" size. And thus, given that, I feel that we can't have a grow-as-you-see-fit buffer. And given that we can't have a grow-at-all buffer; because what is the point if you can't grow to the size you need?

Which leaves us back where we started; almost.

I have created a hybrid of _ReadEncodedStringWithBuffer, but rather than falling through, It does an Append(char[],...) to the StringBuilder, and then reuses the buffer.

I have left the 33-byte buffer, although I think it would be better to increase it, and really I think the size of these buffers should be ultimately pushed up to the user (as well and the management of the char[] and StringBuilder). But I think these can be future changes, and these is value with the current changes without these options.

So hopefully these two static buffer solutions can be accepted, and we can move on.

Below are results with the buffer set at different values (obviously the 500 is the best as that is the max size of created strings in the test; I was quite surprised to see the drop off in performance when the buffer was increased to 1000; but this seemed to be consistent across runs. I haven't tried running the test again where it created bigger strings, and see how that goes...)

Master

Deserializing
=============
Single:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.0258 0.0312
Answer 0.1086 0.1378
Question 0.7215 0.8549

Lists:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.1494 0.2079
Answer 0.4515 0.5568
Question 2.3984 2.7299

Dictionaries of strings to:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.7585 1.02535
Answer 2.64245 3.3555
Question 10.202 13.587

public const int CharBufferSize = 33;

Single:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.0232 0.0312
Answer 0.1008 0.1352
Question 0.6764 0.8428

Lists:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.1396 0.2007
Answer 0.4149 0.5475
Question 2.2542 2.6424

Dictionaries of strings to:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.71725 1.03095
Answer 2.48155 3.32735
Question 10.3535 13.4783

public const int CharBufferSize = 100;

Deserializing
=============
Single:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.0223 0.0303
Answer 0.0954 0.1356
Question 0.6367 0.84085

Lists:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.1383 0.2005
Answer 0.3944 0.5479
Question 2.16965 2.6375

Dictionaries of strings to:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.6896 1.01665
Answer 2.3633 3.3176
Question 10.01745 13.27595

public const int CharBufferSize = 250;

Deserializing
=============
Single:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.0218 0.0312
Answer 0.0919 0.1356
Question 0.6206 0.8482

Lists:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.1311 0.2021
Answer 0.3832 0.551
Question 2.0927 2.7602

Dictionaries of strings to:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.6773 1.0146
Answer 2.32115 3.35525
Question 9.4492 13.6385

public const int CharBufferSize = 500;

Deserializing
=============
Single:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.02 0.0307
Answer 0.0879 0.1356
Question 0.5796 0.8442

Lists:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.1278 0.2012
Answer 0.36385 0.5492
Question 1.94055 2.63195

Dictionaries of strings to:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.6586 1.0169
Answer 2.1692 3.32295
Question 8.24555 13.34065

public const int CharBufferSize = 1000;

Deserializing
=============
Single:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.0205 0.0307
Answer 0.0887 0.1352
Question 0.5992 0.8406

Lists:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.128 0.2007
Answer 0.37185 0.5488
Question 2.0155 2.6576

Dictionaries of strings to:
Type Jil Static MS Jil Static Jil Dynamic MS Jil Dynamic
User 0.6492 1.00085
Answer 2.21385 3.3207
Question 9.5918 13.3433