Extension handling rewrite + cite extension refactoring.

Tokenization
------------
* Fixed tokenizer to correctly parse extension tags in different
  contexts: with start and end tags, in self-closing tag mode,
  and to correctly handle scenarios when the exension end-tag is
  followed by a '#' (the special char used to skip over extension
  content).

* Removed the distinction in the tokenizer between installed
  extensions and natively supported extension tags (<ref> and
  <references> for ex.).  They all tokenize and get processed
  identically and get handled by different paths in the extension
  handler.

* Template and TemplateArg tokens now carry tpl. transclusion
  source alongwith them since tsr information will not be
  accurate when they show up in extension contexts that in turn
  showed up in template context that were expanded by the php
  preprocessor.

  Ex: {{echo|<ref>{{echo|foo}}</ref>}}

  The tsr for the ref-tag will correspond to the
  template-source of the echo-template, NOT the original top-level
  page. So, env.page.src.substring(..) will return incorrect
  source for the innermost {{echo|foo}}. This fix of carrying
  along tpl transclusion source in the token itself eliminates
  this problem.

Knowledge of native extensions
------------------------------
* Natively implemented extension tags (<ref> and <references>)
  are hardcoded in env.conf.parsoid.  At some point, it would
  be good to have a registration mechanism for parsoid-native
  extensions.

Extension handling
------------------
* Extracted extension handling out of the template handler into
  its own handler class. Right now, this class inherits from the
  template handler in order to be able to reuse a lot of the
  expansion and encapsulation functionality currently in the
  Template Handler.

* This handler now handles extensions that are:
  (a) natively implemented and registered with Parsoid.
  (b) implemented as a PHP extension and expanded by relying on
      the PHP preprocessor.

  For (a), it uses information from env.conf.parsoid to find
  ext-handlers for natively implemented ext-tags.  However, this
  can be cleaned up at some point by making available a registration
  mechanism.

Cite/Ref/References
-------------------
* Reworked the cite handler to split up ref-token processing
  and references token processing.
  * The handler now processes ref-tokens, parses content all
    the way to html output and encapsulates the html in an
    attribute of a meta-token that serves as a placeholder for
    where the ref-token occured.
  * References are handled as a DOM post-pass where these meta
    placeholder tokens are collected, content extracted from
    the attribute and spit out at the site of a references tag.
    The DOM walking is in DOMPostProcessor.js, but the actual
    processing is part of the Cite.js to keep all cite extension
    handling code in one place.

Parser pipeline
---------------
* Restructured parser pipeline recipes based on changes to Cite,
  TemplateHandler, and ExtensionHandler.

* Added a couple functions to the parser pipeline:
  1. resetState to reset state before starting a new top-level parse
     when pipelines are reused across top-level parses (ex: parser
     tests)
  2. setSourceOffsets to set start/end offsets of the source being
     handled by the pipeline.  This is required to correctly set tsr
     values when extension content (which is a substring of original
     top-level text) is parsed in its own pipeline.

Other fixes
-----------
* Removed env parameter from the Params object since it was not
  being used and seemed like unnecessary state propagation.
* Removed a FIXME in DOMUtils.buildTokensFromDOM by reusing code
  in the tokenizer that converts "\n" in text to NlTks.
* Cleanup of Util.shiftTokenTSR.
* ext.util.TokenCollection is now no longer used by anything.
  Added a FIXME and left around in case we are able to improve
  tokenizing and handling of *include* tags that can eliminate the
  need for the messy TokenAndAttrCollector.

Test results
------------
* No change in parser tests results.

* Tested with a few different files.
  - en:BO page seems to be parsed about 10% faster than before
    (needs verification).
  - Referencs on the en:BO page seem to be more accurate than
    before.

Change-Id: I8a095fa9fa976c7b3a2a4bd968dc9db4270b105f
Merge "Extension handling rewrite + cite extension refactoring."