In VFPT, we started by using Parser.fs from FSharp.CompilerBinding and then moved to use FCS LineTokenizer. The biggest issue is to fetch correct lexer states. Using `0L` is not correct from multiline comments and strings, etc. You probably have to cache lexing information to be able to do it correctly.