caskdata/hadoop_cookbook — pull request #127 by mandrews (2014-12-03 23:49:35+00:00)
Title: Add tez

Add `tez` recipe following [HDP instructions](http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.5/bk_installing_manually_book/content/rpm-chap-tez.html). Enable `hive` on `tez` by setting:
```ruby
default['hive']['hive_site']['hive.execution.engine'] = 'tez'
```

Example spec run:
```bash
% rspec spec/tez_spec.rb
ffi-yajl/json_gem is deprecated, these monkeypatches will be dropped shortly

hadoop::tez
on Centos 6.5 x86_64
add HDP-2.x yum_repository
add Updates-HDP-2.x yum_repository
add HDP-UTILS-1.1.0.19 yum_repository
install tez package
executes execute[tez-hdfs-appdir]
executes execute[hive-hdfs-appdir]
executes execute[update tez-conf alternatives]
creates /etc/tez/conf.chef directory
creates /etc/tez/conf.chef/tez-site.xml template
creates /etc/tez/conf.chef/tez-env.sh template

Finished in 1.57 seconds (files took 1.28 seconds to load)
10 examples, 0 failures

ChefSpec Coverage report generated...

Total Resources: 9
Touched Resources: 9
Touch Coverage: 100.0%

You are awesome and so is your test coverage! Have a fantastic day!
```

Task for `rake foodcritic` and `rake rubocop` are passing.

Example default `hive` execution engine run:
```bash
hive> select count(1) from foo;
Query ID = hdfs_20141203043030_be0e24d2-5f7e-4450-a169-4200b354ec1e
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=
In order to set a constant number of reducers:
set mapreduce.job.reduces=
Starting Job = job_1417580222608_0002, Tracking URL = http://dw1.local:8088/proxy/application_1417580222608_0002/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1417580222608_0002
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2014-12-03 04:31:00,828 Stage-1 map = 0%, reduce = 0%
2014-12-03 04:31:08,414 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 0.94 sec
2014-12-03 04:31:16,785 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 2.32 sec
MapReduce Total cumulative CPU time: 2 seconds 320 msec
Ended Job = job_1417580222608_0002
MapReduce Jobs Launched:
Job 0: Map: 1 Reduce: 1 Cumulative CPU: 2.32 sec HDFS Read: 257 HDFS Write: 2 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 320 msec
OK
0
Time taken: 29.014 seconds, Fetched: 1 row(s)
```

Example `tez` `hive` execution engine run:
```bash
hive> select count(1) from foo;
Query ID = hdfs_20141203042929_bd9a75fc-c4ff-48cc-b4ff-51407f8fb649
Total jobs = 1
Launching Job 1 out of 1

Status: Running (application id: application_1417580222608_0001)

Map 1: -/- Reducer 2: 0/1
Map 1: -/- Reducer 2: 0/1
Map 1: -/- Reducer 2: 0/1
Map 1: -/- Reducer 2: 1/1
Status: Finished successfully
OK
0
Time taken: 12.973 seconds, Fetched: 1 row(s)
```