Hadoop 2.3 dev (pebbles) Submitter: Paul Webster Reviewer: Vinayak Ponangi
  * Cloudera Manager BC - fred to pebbles code sync.
Merge pull request #110 from Paul-Webster/pull-req-release-pebbles-master-24dc7bee22558faf0bfdeb8a536f6ce4b04fa45e

Cloudera Manager BC - fred to pebbles code sync [1/1]
Update crowbar.yml files for redhat and centos 6.4
Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
   * Upgraded the RESTful API client version to v3. CM 4.5 implements the API functionality at
     a v3 level and this change improves functional capabilities of the RESTful client interface.
   * Added code to programmatically create/set the CM services, roles and host instances (cm_api.rb).
     This code is fully debugged but the host instance information is still statically coded in an
     in-line data structure for debugging purposes (live cluster environment).
     We still need to write code to populate this data structure from the crowbar wall and
     Chef role assignments - working on that next.
   * Added support to set the CM "rack_id" parameter via the API.
   * Class name fix up in BaseApiObject - cls.from_json_dict(). It was incorrect (cls.class.name->cls).
   * Added roleConfigGroupRef RO_ATTR to the APIRoleObject. This ATTR is an API v3 dependency and
     required implementation to properly support the API at the v3 level.
   * CM_API code execution is still disabled by default via dev mode flag - "deployment_type=manual".
     It will remain that way until the functional aspects of CB/CM API integration is fully debugged.
Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
  * Fixed up file permissions.
Merge pull request #111 from Paul-Webster/pull-req-release-pebbles-master-1cee0b5285c28fb3425a26007b2f25da035b6d8f

Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi [1/1]
Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
   * Added the to_safe_url() method to libraries/resource.rb (Resource Object)
     in order to allow the RESTful API base URL to contain embedded spaces
     (i.e. CM cluster name = "Cluster 1 - CDH 4").
     Patched the new method call into POST, PUT, GET and DELETE.
   * Removed the cm_server text field from the Crowbar Cloudera Manager proposal
     because this node location is explicitly determined by Crowbar role
     assignment and the field is not required.
Merge pull request #112 from Paul-Webster/pull-req-release-pebbles-master-5d296978cf96a0cd837f2d228fdc5af262c64330

Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi [1/1]
Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
   * Code changes relating to setting up CM hosts, roles and services.
     Mostly fixes to the code which was already implemented.
   * Added code to populate the cluster_config data structure from Crowbar role definitions.
     This data structure was fixed coded before to allow for API method testing.
     It is now dynamically populated from crowbar role definitions as it should be.
     The data is derived from node-setup.rb.
     node_rec  = { :fqdn => n[:fqdn], :ipaddr => ipaddr, :name => n.name, :ssh_key => ssh_key }
   * Refactored the code in cm-api.rb. Moved away from an in-line approach and more towards a
     discrete subroutine based implementation.
     This provides a more consistent overall framework for additional high level flow control
     and error handling (which I plan to enhance in the future).
   * Split default.rb into 2 separate files (node-setup.rb and hadoop-setup.rb) in order to
     isolate crowbar functionality v.s. hadoop functionality.
     This was needed in order to segregate code execution from the crowbar admin node v.s. hadoop cluster nodes.
     default.rb does not exist anymore.
   * Eliminated some chef node search calls. The CM_API requires much more querying of the cluster
     topology and the code was getting un-wieldy. The CB role definitions are now parsed out up front
     and this role/node information is store as an array on the Chef node object (See node-setup.rb).
   * All code tested at the unit and system level on a live 8 node Hadoop cluster.
   * CM_API code execution is still disabled by default via dev mode flag - "deployment_type=manual".
     It will remain that way until the functional aspects of CB/API execution is fully debugged.
   * Diffs will be big due to the code reallocation. Let me know if you have any questions.
Merge pull request #113 from Paul-Webster/pull-req-release-pebbles-master-cb0d5cc0a600ada0ab95675d5ccf7c3e8540058c

Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi [1/1]
Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
   * Fixed an issue with SSH keys not being updated properly.
Merge pull request #114 from Paul-Webster/pull-req-release-pebbles-master-b12505f8d863e262fb52d1c72fd449e48571ca24

Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi [1/1]
Hadoop 2.3/pebbles/RHEL 6.4 - CM API role assignment tuning - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
   * Refactored CM role assignment code in order to correctly match CB->CM roles.
   * Added the CM GATEWAY role assignment to the CB edgenodes.
   * Added code to automatically re-start the cm-server if the license key is updated (or changed).
   * Refactored the API client get_license() method to return nil if no license key found (RestClient::ResourceNotFound exception rescued).
   * All code tested at the unit and system level on a live 8 node Hadoop cluster.
   * CM_API code execution is still disabled by default via dev mode flag - "deployment_type=manual".
     It will remain that way until the functional aspects of CB/API execution is fully debugged.
Merge pull request #115 from Paul-Webster/pull-req-release-pebbles-master-dcc26aaf8a51c03d769518128754bbb5dc70d98c

Hadoop 2.3/pebbles/RHEL 6.4 - CM API role assignment tuning - Submitter: Paul Webster  Reviewer: Vinayak Ponangi [1/1]
Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
   * Added code to properly configure the MAPREDUCE service role.
   * Added code to properly configure the JOBTRACKER service role.
   * Added code to properly configure the CM GATEWAY service role.
   * All code tested at the unit and system level on a live 8 node Hadoop cluster.
   * CM_API code execution is still disabled by default via dev mode flag - "deployment_type=manual".
     It will remain that way until the functional aspects of CB/API execution is fully debugged.
Merge pull request #116 from Paul-Webster/pull-req-release-pebbles-master-d807fad3ceccfa5e347dc32709b23c4ec69df4dd

Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi [1/1]
Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
   * Dropped the API version back to v2 because the v3 API has enhanced functional requirements
     which impact our RESTful API client code implementation (e.g.  CM role groups must now be used
     when running at the v3 level) The API client needs some work in order to support this
     functionality (TBD). The enhanced v3 feature set is not required for this release and the v2 API
     maintains backwards compatibility with our RESTful client code implementation.
   * Code debugging/fixes in regards to setting up service level configuration parameters
     (Required to get the base HDFS/MAPR services running).
   * All code tested at the unit and system level on a live 8 node Hadoop cluster.
   * CM_API code execution is still disabled by default via dev mode flag - "deployment_type=manual".
     It will remain that way until the functional aspects of CB/API execution is fully debugged.
   * Diffs will be big due to the code reallocation. Let me know if you have any questions.
Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi
   * Reworked the service level control layer (services.rb) so it is
     parented through the top level ApiResource class.
   * Added service level control entry point methods to ApiResource.
   * Added mainline code to set required MAPR configuration parameters.
   * Added mainline code to cm-api.rb to format HDFS and automatically start the
     HDFS service (still has a few kinks in it and needs further debugging - code
     block commented out on commit).
   * All other code tested at the unit and system level on a live 8 node Hadoop cluster.
   * CM_API code execution is still disabled by default via dev mode flag - "deployment_type=manual".
     It will remain that way until the functional aspects of CB/API execution is fully debugged.
Merge pull request #117 from Paul-Webster/pull-req-release-pebbles-master-2fa93fe1679c0909016db51fa11c7696afa159ef

Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Vinayak Ponangi [1/1]
Merge pull request #118 from Paul-Webster/pull-req-release-pebbles-master-8a56258ad3d46b483bb3f09b29a9e267ad163aeb

Hadoop 2.3/pebbles/RHEL 6.4 - CM API dev work - Submitter: Paul Webster  Reviewer: Jim Clement [1/1]