Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi [1/1]

Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
   * Code changes relating to setting up CM hosts, roles and services.
     Mostly fixes to the code which was already implemented.
   * Added code to populate the cluster_config data structure from Crowbar role definitions.
     This data structure was fixed coded before to allow for API method testing.
     It is now dynamically populated from crowbar role definitions as it should be.
     The data is derived from node-setup.rb.
     node_rec  = { :fqdn => n[:fqdn], :ipaddr => ipaddr, :name => n.name, :ssh_key => ssh_key }
   * Refactored the code in cm-api.rb. Moved away from an in-line approach and more towards a
     discrete subroutine based implementation.
     This provides a more consistent overall framework for additional high level flow control
     and error handling (which I plan to enhance in the future).
   * Split default.rb into 2 separate files (node-setup.rb and hadoop-setup.rb) in order to
     isolate crowbar functionality v.s. hadoop functionality.
     This was needed in order to segregate code execution from the crowbar admin node v.s. hadoop cluster nodes.
     default.rb does not exist anymore.	
   * Eliminated some chef node search calls. The CM_API requires much more querying of the cluster
     topology and the code was getting un-wieldy. The CB role definitions are now parsed out up front
     and this role/node information is store as an array on the Chef node object (See node-setup.rb).
   * All code tested at the unit and system level on a live 8 node Hadoop cluster.
   * CM_API code execution is still disabled by default via dev mode flag - "deployment_type=manual".
     It will remain that way until the functional aspects of CB/API execution is fully debugged.
   * Diffs will be big due to the code reallocation. Let me know if you have any questions.




 .../clouderamanager/attributes/default.rb          |    4 +
 chef/cookbooks/clouderamanager/recipes/cm-agent.rb |   18 +-
 chef/cookbooks/clouderamanager/recipes/cm-api.rb   |  232 +++++++++++------
 .../cookbooks/clouderamanager/recipes/cm-server.rb |    8 +-
 chef/cookbooks/clouderamanager/recipes/default.rb  |  265 --------------------
 .../clouderamanager/recipes/hadoop-setup.rb        |  232 +++++++++++++++++
 .../clouderamanager/recipes/node-setup.rb          |  136 ++++++++++
 chef/roles/clouderamanager-cb-adminnode.rb         |    1 +
 chef/roles/clouderamanager-datanode.rb             |    3 +-
 chef/roles/clouderamanager-edgenode.rb             |    3 +-
 chef/roles/clouderamanager-ha-filernode.rb         |    3 +-
 chef/roles/clouderamanager-ha-journalingnode.rb    |    3 +-
 chef/roles/clouderamanager-namenode.rb             |    3 +-
 chef/roles/clouderamanager-server.rb               |    3 +-
 14 files changed, 552 insertions(+), 362 deletions(-)

Crowbar-Pull-ID: cb0d5cc0a600ada0ab95675d5ccf7c3e8540058c

Crowbar-Release: pebbles