Instalação Hadoop e Mahout no Linux CentOS
Autor: Cleydson Silva | Publicado: 2014-12-10T23:17:00-02:00



Baixar a versão mais nova do Hadoop

http://ftp.unicamp.br/pub/apache/hadoop/common/
Neste momento a versão mais nova está no link: http://ftp.unicamp.br/pub/apache/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz

Comandos:
$ wget http://ftp.unicamp.br/pub/apache/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz
$ tar -zxvf hadoop-1.2.1.tar.gz
$ sudo cp -r hadoop-1.2.1/ /usr/local


Instalar o Java SDK

No Linux CentOS podemos usar o yum. Na versão atual do Java SDK o

Comandos:&nbsp;
$ yum install java-1.7.0-openjdk-devel
ou
$ yum install java-devel


Baixar a versão mais nova do&nbsp;Mahout
http://ftp.unicamp.br/pub/apache/mahout/
Neste momento: http://ftp.unicamp.br/pub/apache/mahout/0.8/mahout-distribution-0.8.tar.gz

Comandos:
$ wget http://ftp.unicamp.br/pub/apache/mahout/0.8/mahout-distribution-0.8.tar.gz
$ tar -zxvf mahout-distribution-0.8.tar.gz
$ sudo cp -r mahout-distribution-0.8/ /usr/local

Adicionar as linhas abaixo no Arquivo&nbsp;~/.bachrc:

export HADOOP_PREFIX=/usr/local/hadoop-1.2.1
export PATH=$PATH:$HADOOP_PREFIX/bin
export MAHOUT_HOME=/usr/local/mahout-distribution-0.8
export PATH=$PATH:$MAHOUT_HOME
export JAVA_HOME=/usr/lib/jvm/java-1.6.0

Configurando o ssh para conexão de um servidor no outro sem pedir a senha:

$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
$ cat ~/.ssh/id_dsa.pub &gt;&gt; ~/.ssh/authorized_keys

Configurando o Cluster
No master:
/etc/hosts adicionar:
192.168.0.123 &nbsp;master
192.168.0.131 &nbsp;slave1
192.168.0.132 &nbsp;slave2

$HADOOP_HOME/conf/master adicionar:
master

$HADOOP_HOME/conf/slaves adicionar:
master
slave1

Em todas as máquinas:

conf/core-site.xml

&lt;property&gt;
&lt;name&gt;fs.default.name&lt;/name&gt;
&lt;value&gt;hdfs://master:54310&lt;/value&gt;
&lt;description&gt;The name of the default file system. A URI whose&nbsp;scheme and authority determine the FileSystem implementation. The&nbsp;uri's scheme determines the config property (fs.SCHEME.impl) naming&nbsp;the FileSystem implementation class. The uri's authority is used to&nbsp;determine the host, port, etc. for a filesystem.&lt;/description&gt;
&lt;/property&gt;

conf/mapred-site.xml

&lt;property&gt;
&lt;name&gt;mapred.job.tracker&lt;/name&gt;
&lt;value&gt;master:54311&lt;/value&gt;
&lt;description&gt;The host and port that the MapReduce job tracker runs at. If "local", then jobs are run in-process as a single map and reduce task.
&lt;/description&gt;
&lt;/property&gt;

conf/hdfs-site.xml

Atenção:&nbsp;&lt;value&gt;2&lt;/value&gt;&nbsp;abaixo refere-se a quantidade de nós disponíveis.

&lt;property&gt;
&lt;name&gt;dfs.replication&lt;/name&gt;
&lt;value&gt;2&lt;/value&gt;
&lt;description&gt;

Default block replication.&nbsp;The actual number of replications can be specified when the file is created.&nbsp;The default is used if replication is not specified in create time.
&lt;/description&gt;
&lt;/property&gt;

Logo depois executar

Comando:
$ start-all.sh

Links:
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/
http://www.higherpass.com/linux/Tutorials/Installing-And-Using-Hadoop/1/
http://wiki.apache.org/hadoop/PoweredBy