Upgrade to ExtCore 0.8.40 to take advantage of some performance improvements.
Implemented some small optimizations; instead of using tuples as cache keys, use nested maps.
Implemented some new types (SourcePositionRange and PositionedValue) to clean up the way we handle propagation of source position information.
Implemented caching for sets of derivative classes for Regexes. This seems to have improved performance, although the F# compiler lexer still takes an extremely long time to compile -- most of the compilation time is spent in the CharSet comparison functions due to comparing equal (but not identical) CharSets for equality, so implementing hash-consing for CharSets might provide enough of a performance gain to allow the F# compiler lexer to compile in a reasonable amount of time.
Implement functionality to ignore specific test cases (which is sometimes useful during the development process).
Implemented some small optimizations in the CharDiet type.
Split some of the files into smaller files so the code can be re-ordered for the purposes of introducing a new record type (in Caching.fs) to streamline the way we handle caching (memoization) throughout the code.
Modified the code to use the new CompilationCache record type instead of manually passing around the individual caches.
More aggressive interning of CharSets; this provides a large performance increase because it eliminates a significant amount of time spent comparing equal (but not identical) CharSet instances.
Intern the derivative classes before trying to find the cached intersection.
Add the intersection of derivative classes to the cache after computing it.
Added a pair of rewrite rules to the smart constructor for the Or pattern, to compact an Or pattern where the two sub-patterns are Kleene stars of character sets. This also seems to have fixed the termination problem which has been plaguing fsharplex for a long while.
Use direct recursion instead of the mutable stacks; although they guarantee stack overflows won't happen, the maximum depth of a CharDiet is fairly small due to the limited range of the `char` type. Direct recursion has also been empirically shown to be faster.
Increase the test timeout from 30 seconds to 120; the F# lexer test should complete in under 30 seconds on recent (e.g., Nehalem or newer) machines, so the additional timeout is just to help keep the test from timing-out on older machines.
Renamed FSharpLex.Tests/Diet.fs to FSharpLex.Tests/Collections.fs.
Fixed the DerivativeClasses.intersect function so it sorts the keys (the two DerivativeClasses used as inputs to the function) to increase the cache hit ratio and reduce the size of the cache.
Removed now-redundant caching from the RegularVector.derivativeClasses function (the caching logic was previously moved into the DerivativeClasses.intersect function).