crowbar/barclamp-hadoop_infrastructure — pull request #7 by Paul-Webster (2013-12-31 16:16:28+00:00)
Title: Implemented JJ.s fix for c8k/2 2.5 inch drives in the Hadoop 2.4 code branch (CES-20). [1/2]

Implemented JJ.s fix for c8k/2 2.5 inch drives in the Hadoop 2.4 code branch (CES-20).
Ported the code patch from the mesa-1.6.1 code branch to the hadoop 2.4 (some modifications required).

Added this method in the hadoop 2.4 code branch:

~/crowbar/barclamps/deployer/chef/cookbooks/barclamp/libraries/barclamp_library.rb

def is_ahci
Chef::Log.info("Testing whether #{@device} is an ahci dev")
res = false
d_path = "/sys/block"
name = File.join(d_path,@device)
return res unless File.symlink?(name)
link = File.readlink(name)
path = File.expand_path(link,d_path)
#get the last pci path
p_pci = path.gsub( /(.*[0-9a-f]+:[0-9a-f]+:[0-9a-f]+\.[0-9a-f]+).*/, '\1')
#Check if the pci driver is AHCI
p_pci_driver = File.join(p_pci,'driver')
driver_link = File.readlink(p_pci_driver)
res = true if driver_link =~ /ahci$/
Chef::Log.info("#{@device} is an ahci dev") if res
res
end

How the patch was previously implemented in the openstack/mesa-1.6.1 code branch (disks on the cb wall):

~/mesa-1.6.1/barclamps/swift/chef/cookbooks/swift/recipes/disks.rb

to_use_disks = []
all_disks = eval(node[:swift][:disk_enum_expr])
all_disks.each { |k,v|
b = binding()
# skip if the disk is ahci and we have raid.
next if v.is_ahci and node[:crowbar_wall].has_key?('raid')
to_use_disks << k if eval(node[:swift][:disk_test_expr]) && ::File.exists?("/dev/#{k}")
}

How I implemented the changes in the Cloudera infrastructure barclamp (hadoop-2.4 . using the disk claim procedure).

Note: There is no "raid" key on the crowbar wall for the hadoop 2.4 and this
was removed as it is not necessary in the Hadoop case. This code only executes on
Hadoop datanodes and all AHCI enabled drives are skipped.

~/crowbar/barclamps/hadoop_infrastructure/chef/cookbooks/hadoop_infrastructure/recipes/configure-disks.rb

#----------------------------------------------------------------------
# Find all the unclaimed disks and claim them (Hadoop 2.4 or higher).
#----------------------------------------------------------------------
BarclampLibrary::Barclamp::Inventory::Disk.unclaimed(node).each do |disk|
# Skip all ahci enabled disks on the datanodes.
next if disk.is_ahci
# Reserve disk usage for Cloudera.
if disk.claim("Cloudera")
Chef::Log.info("Claiming #{disk.name} for Cloudera")
else
Chef::Log.info("Failed to claim disk #{disk.name} for Cloudera")
end
end

.../recipes/configure-disks.rb | 11 +++++++----
1 files changed, 7 insertions(+), 4 deletions(-)

Crowbar-Pull-ID: fd800aafdb5fb98b4bb78cf719a6b279520a47cd

Crowbar-Release: hadoop-2.4