Import Hadoop barclamp
Dos2unix, Part 4
Make the Hadoop default recipe pull in the Oracle JDK and erase the openjdk.
Code patches to fix redhat deployment issues.
Added "crowbar-committing", "crowbar-queued" to hadoop schema (these got
dropped from the allocate state code refactor).
Added code to automatically compute cluster node distribution on default
proposal commit.
Reworked the node distribution logic to only add the edge node if the
node count is greater than 3.
DE225: rename openstack_manager to crowbar_framework and remove openstack
Hadoop barclamp bug fixes:
DE244 - FIXED - Oracle JVM is not installed on the compute nodes.
DE219 - FIXED - Description for Hadoop barclamp references ecosystem
components.
Merge branch 'dell-hadoop' of 10.9.244.31:crowbar into dell-hadoop
Changes to properly support Cloudera's SCM enterprise edition tools
under redhat 5.6.
Merge branch 'dell-hadoop' of 10.9.244.31:crowbar into dell-hadoop
Fixed the code which writes the cluster config information
(/etc/hadoop/conf/masters & /etc/hadoop/conf/slaves).
Fixed the code which sets core:fs_default_name (not executing on the
slave nodes - moved to default.rb).
Fixed type in code - core-site.xml was not being written properly.
Removed hadoop sub-packages from the hadoop barclamp.
Removed sub-package map references from the json & schema files.
Moving sub-package functionality to separate barclamps.
The hadoop barclamp now only supports baseline hadoop installation
functionality (HDFS/Map Reduce).
Removed hadoop sub-packages from the hadoop barclamp.
Removed sub-package map references from the json & schema files.
Moving sub-package functionality to separate barclamps.
The hadoop barclamp now only supports baseline hadoop installation
functionality (HDFS/Map Reduce).
Hadoop Barclamp - default values need to be updated to RA specification.
Review with AD and made the following adjustments;

From: Dumitru, Aurelian
Sent: Wednesday, August 31, 2011 4:38 PM
To: Webster, Paul
Subject: [MODIFICATIONS] hadoop bc proposal attached - pls check/adjust
the values against the RA

Paul,

Following entries need to be replaced/updated:

"dfs_block_size": " 134217728",
"dfs_datanode_du_reserved": " 10737418240",
"dfs_namenode_handler_count": "32",
"dfs_datanode_handler_count": "16",
"fs_trash_interval": "1440",
"mapred_job_tracker_handler_count": "32",
"mapred_reduce_tasks": "6",
"mapred_tasktracker_map_tasks_maximum": "10",
"mapred_tasktracker_reduce_tasks_maximum": "6",
"mapred_child_java_opts": "- -Xmx1024m",
"mapred_child_ulimit": " 2097152",
"mapred_map_tasks_speculative_execution": "false",
"mapred_reduce_tasks_speculative_execution": "false",
"io_file_buffer_size": " 65536",
"io_sort_factor": "80",
"io_sort_mb": "512",
Remove the packaging files.
Move the view files into the barclamp.
DE 272 - Push my hadoop UI proposal changes back to git.
They seem to have gotton lost somehow.
Removed the hadoop macro syntax (i.e. ${haddop.tmp.dir}) from the
configuration parameters and specified the paths directly. These macros
were casing configuration problems with Map/Reduce.
I made notes of these adjustments in the code - search for "DEFAULT:" to
locate the changes.
DE272, DE286, DE 271, DE276 and other fixes relating to Hadoop
Map/Reduce.
DE 288 is a dup of DE 278 and Im closing it out.
Other changes;
	Changes to set the directory ownership for map/reduce properly
(mapred/hdfs).
	Adjusted a few hadoop parameter settings to RA values (feedback
from AD & Joey).
	Set the jobtracker and jobtracker.http address:port to the
master node IP (feedback from Joey).
DE291 Fixed - Hive incorrectly escaped value.
Added HAML code to render the custom attribute and deployment widgets.
Added the model hive_service.rb code to bind the proposal to edge nodes.
Note - This barclamp is still in a dev testing state.
Please test - but expect some issues.