D-Programming-Language/dmd — pull request #2470 by hpohl (2013-08-12 22:41:55+00:00)
Title: [experimental] guess how much memory will be needed and adjust chunk size

The guessed memory is adequate to the total file size of sources given. I don't think it's a good idea but the results are rather good.

Benchmarked using `avgtime -d -r 10 dmd std/algorithm -unittest -o-`.

Default DMD:
```
Total time (ms): 60364.4
Repetitions : 10
Sample mode : 6190 (1 ocurrences)
Median time : 6057.17
Avg time : 6036.44
Std dev. : 127.367
Minimum : 5769.23
Maximum : 6196.66
95% conf.int. : [5786.8, 6286.07] e = 249.634
99% conf.int. : [5708.36, 6364.51] e = 328.075
EstimatedAvg95%: [5957.5, 6115.38] e = 78.9412
```
Guessing DMD:
```
Total time (ms): 51912.9
Repetitions : 10
Sample mode : 5170 (2 ocurrences)
Median time : 5167.91
Avg time : 5191.29
Std dev. : 139.772
Minimum : 5030.62
Maximum : 5520.05
95% conf.int. : [4917.34, 5465.23] e = 273.947
99% conf.int. : [4831.26, 5551.31] e = 360.028
EstimatedAvg95%: [5104.66, 5277.92] e = 86.6297
EstimatedAvg99%: [5077.44, 5305.14] e = 113.851
```

Even though the guess/needed ratio is 0.07 so 7% of the memory needed is preallocated, there is a ~16% speedup. The allocated/needed ratio is at ~1.0054 which is quite good. There are some printfs you can use to adjust the magic numbers or to try different source files.