Tokenization

+ Convert a string into a list of tokens. Before this occurs, we have to
  invoke the preprocessor and remove all comments (NIY). After this list
  is obtained, it could be further refined (e.g., differentiate between
  types and other identifiers; differentiate between template <> and
  other occurrences of <>, ...) and a tree has to be built from the
  list (both NIY)
  Also NIY is support for floating point literals (although they are
  probably recognized as [integer literal] [operator '.'] [integer
  literal] [identifier 'e'] [operator '+'/'-'] [integer literal] or
  something like that).