PhilippeSigaud/Pegged — pull request #78 by callumenator (2012-10-30 18:55:09+00:00)
Title: NOT asking you to merge this....

just wanted to let you know about it, this seemed the easiest way.

Been playing around with a grammar for roughly tokenizing D code, want to use it for a syntax highlighter. I have been running it on the std.string module (3728 loc) to get a sense of performance, and have been optimizing the low-hanging fruit in peg.d (in this perfo branch). Here are some timings, each number is an average of 10 runs, with and without memoization (all in msecs):

git head (Memo.no): 4405, 4046, 4307
git head (Memo.yes): 5013, 4840, 4868

perfo (Memo.no): 941, 840, 929
perfo (Memo.yes): 1692, 1506, 1535

The speed difference regarding memoization is interesting, I haven't looked into it any further.

Most of the optimizations are just avoiding string concats when they can be done at compile time (the ParseTree names for example are a hotspot). I also tried replacing the keywords if-chain with a CT-generated switch using a trie (ripped out of Dscanner, thanks Brian!), but that did not have as much of an impact as I thought it would.

There are still places where performance can be improved, I just hacked at the easy stuff identified by a profiler. Let me know what you think.

Cheers
Cal