CDH-13042: add morphline support to HBase indexer

/**
 * Pipes a given HBase Result into a morphline and extracts and transforms the specified HBase cells
 * to a SolrInputDocument. Note that for proper functioning the morphline should not contain a
 * loadSolr command because loading documents into Solr is the responsibility of the enclosing Indexer.
 * 
 * Example config file:
 * 
 * <pre>
 * <indexer 
 * 
 *   <!--
 *   The HBase Lily Morphline Indexer supports the standard attributes of an HBase Lily Indexer 
 *   (i.e. table, mapping-type, read-row, unique-key-formatter, unique-key-field, row-field, column-family-field), 
 *   as documented at https://github.com/NGDATA/hbase-indexer/wiki/Indexer-configuration
 *
 *   In addition, morphline specific attributes are supported, as follows:
 *   --> 
 * 
 *   <!-- The name of the HBase table to index -->
 *   table="record" 
 *   
 *   <!-- Parameter mapper (required): Fully qualified class name of morphline mapper -->
 *   mapper="com.ngdata.hbaseindexer.morphline.MorphlineResultToSolrMapper">
 *   
 *   <!--
 *   Parameter morphlineFile (required): The relative or absolute path on the local file system to the morphline configuration file. Example: /etc/hbase-solr/conf/morphlines.conf
 *   -->
 *   <param name="morphlineFile" value="morphlines.conf"/>
 *   
 *   <-- 
 *   Parameter morphlineId (optional): Name used to identify a morphline if there are multiple morphlines in a morphline config file 
 *   -->
 *   <param name="morphlineId" value="morphline1"/>
 *   
 * </indexer>
 * </pre>
 */


morphlines : [
  {
    id : morphline1
    importCommands : ["com.cloudera.**", "com.ngdata.**"]
    
    commands : [                    
      {
        # The extractHBaseCells morphline command extracts cells from an HBase Result 
        # and transforms the resulting values into a SolrInputDocument.
        # It consists of an array of zero or more mapping specifications.
        extractHBaseCells {
          mappings : [
            {
              # The inputColumn specifies the data to be used from HBase for populating the field in Solr. 
              # It takes the form of a column family name and qualifier, separated by a colon.
              # The qualifier portion can end in an asterisk, which is interpreted as a wildcard. 
              # In this case, all matching column-family and qualifier expressions will be used.
              #
              # The following are examples of valid inputColumn values:
              # 
              # mycolumnfamily:myqualifier
              # mycolumnfamily:my*
              # mycolumnfamily:*
              inputColumn : "cfA:qualifierA"
              
              # The outputField specifies the morphline record field (aka Solr document field) to add output values to
              outputField : fieldA
              
              
              # The type parameter defines the datatype of the content in HBase.
              # Because all data is stored in HBase as byte arrays, but all content in Solr is indexed as text, 
              # a method for converting from byte arrays to the actual datatype is needed. 
              # The type parameter can be the name of a type that is supported by org.apache.hadoop.hbase.util.Bytes.toXXX 
              # (currently: "byte[]", "int", "long", "string", "boolean", "float", "double", "short", bigdecimal"), 
              # or it can be the name of a class that implements the ByteArrayValueMapper interface.
              # Use type "byte[]" to pass the byte array through to the morphline without any conversion.
              type : int
              
              # The source parameter determines what portion of an HBase KeyValue will be used as indexing content. 
              # It has two possible values: "value" or "qualifier".
              # When "value" is specified (which is the case by default), then the cell value is used as input for indexing.
              # When "qualifier" is specified, then the column qualifier is used as input for indexing.
              source : value
            },
            
            {
              inputColumn : "cfB:qual*"
              outputField : fieldB
              type : string
            }
          ]
        }
      }
      
      { logDebug { format : "output record: {}", args : ["@{}"] } }    
    ]
  }
]

For those new to morphlines here is some background:

- Introduction: http://blog.cloudera.com/blog/2013/07/morphlines-the-easy-way-to-build-and-integrate-etl-apps-for-apache-hadoop/

- Reference Guide: http://cloudera.github.io/cdk/docs/current/cdk-morphlines/morphlinesReferenceGuide.html

- Source code: https://github.com/cloudera/cdk/tree/master/cdk-morphlines

BTW, On August 1, CDK 0.5.0 will be released with a new version of morphlines. For example, this will allow to get HTML cells from HBase via the Lily Morphline HBase Indexer, convert the HTML and XML and use XPath/XQuery to extract links, and other metadata from the HTML and index it in Solr - https://github.com/cloudera/cdk/blob/master/src/site/markdown/release_notes.md

Wolfgang.