Print status messages to the console when compiling to give the user an indication of compilation progress.
Now that we know why the generated F# compiler lexer isn't working, add a comment with some additional information about the format required for fslex-compatible Unicode lexer tables.
Split the function for emitting the transition and action tables into several smaller functions to make the code easier to understand.
Moved the functions for emitting the ASCII and Unicode transition vectors into separate modules; later, they'll be broken up into smaller, more maintainable (and testable!) functions.
Moved functions for computing the set of Unicode characters used to label transitions into the UnicodeLexer module.
Optimized the Compile.isAsciiCharSet function.
Remove 'regexAlphabetMapReduce' -- we don't actually need to compute the alphabet for the regex when negating character sets, we'll just use the full range of ASCII or Unicode characters.
Updated comments related to rewriting negated character sets; we don't actually need to do this for fslex compatibility, so the 'rewriteNegatedCharSets' function could be removed. However, it remains for now because removing it exposes a non-termination bug when compiling the F# compiler lexer.
Fixed an issue where the Negate(Epsilon) pattern wasn't being simplified; as a result, we don't need to rewrite negated character sets during compilation.
Fixed the normalization/simplification function for Regex so it correctly handles some patterns which were causing non-termination.
Finished the implementation of the Regex.DebuggerDisplay property so it's easier to see the form of regexes when debugging.
Removed the Any pattern, because it's redundant -- instead, represent the 'Any' pattern as Negate Empty.
Committing here to keep a record of an odd (but not-serious) bug in the F# compiler's pattern-match exhaustivity checker where it's emitting extraneous warnings.
Removed the bit of code which was triggering the extraneous warnings from the F# compiler (the code isn't necessary anyway).
Fixed some issues caused by the removal of the Any pattern (which was replaced with the equivalent Negate Empty); some of these replacements broke the computation of derivative classes, so the generated lexers had exactly 3 states for each rule. These fixes restore the correct behavior.
Make FSharpLex.Graph a namespace instead of a module.
Moved the implementation of charset printing into CharSet.ToString().
Fixed the way transition vectors are emitted for Unicode-based tables. The current implementation isn't efficient, but it works -- fsharplex can now compile the lexer definitions needed to rebuild the F# compiler.
Updated README to remove information about the fsyacc Unicode table-generation bug now that it's fixed.
Changed the Ast.CodeFragment type to be an abbreviation for the ExtCore.Substring type instead of String. This is basically a no-op for now, but it will allow the code fragments to be handled more efficiently going forward.
Implemented the Pattern.Xor case; as with Pattern.And and a few others, the front-end doesn't yet provide syntax for creating this pattern so it's only useful for testing purposes.
Remove a small piece of unused code.
Modified the fslex-compatible backend so it emits code correctly when a custom lexer library namespace is specified on the command line.
Updated README to add a "known issue" regarding fsharplex's current lack of position information in generated lexer code.