CDP私有云安装

# 概要
换工作休假中。趁着家里有网，搞一个私有云试试
## 一些参考资料
私有云下载： https://www.cloudera.com/downloads/cdp-private-cloud-trial/cdp-private-cloud-base-trial.html
官方文档： https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/installation/topics/cdpdc-installing-trial-cluster.html

注意：官方文档的版本是7.17，笔者自己玩的时候已经是7.4.4

## 服务器
```
zzeng01： manager+namenode
zzeng02： datanode
zzeng03： datanode
```

配置都是8cpu 32Gmem

# 安装和卸载

基本按着官网资料做就完事，卸载不要看官网，已经是旧的。

## 安装
参考： https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/installation/topics/cdp-quick-start-deployment-streams-install-runtime.html

```
$ wget https://archive.cloudera.com/cm7/7.4.4/cloudera-manager-installer.bin
$ chmod u+x cloudera-manager-installer.bin
$ sudo ./cloudera-manager-installer.bin
```
hint:为了方便可以设置/etc/hosts

```
[IP]  zzeng01.novalocal
[IP]  zzeng02.novalocal
[IP]  zzeng03.novalocal
```

## 卸载
```
sudo /opt/cloudera/installer/uninstall-cloudera-manager.sh
```

# 配置

访问网址进行修改 http://zzeng01.novalocal:7180/cmf/login

参考： https://docs.cloudera.com/cdp-private-cloud-base/7.1.7/installation/topics/cdp-quick-start-deployment-streams-install-runtime.html

## Failed to authenticate

Install Agents阶段出现Failed to authenticate
做了以下尝试后成功解决问题。

### 设置hosts和hostname

据说cdp内部是通过ip反找hostname，这里需要设置（但最后这里是不是问题的原因未知，总之我是一起修改了。。）

```ruby:在三台服务器都要做
$ sudo cat /etc/hosts
10.113.205.187  ip-10.113.205.187.se.fuse.cloudera.com zzeng01.novalocal
10.113.206.35   ip-10.113.206.35.se.fuse.cloudera.com zzeng02.novalocal
10.113.204.129  ip-10.113.204.129.se.fuse.cloudera.com zzeng03.novalocal
```

```ruby:在三台服务器都要做
$ sudo hostnamectl set-hostname ip-10.113.205.187.se.fuse.cloudera.com
```

### unlock root

centos默认root用户是锁住的，因为我们用root，所以需要解锁

<details>
<summary>新建用户也可以</summary>
<p>在每个server上准备一个新的用户名
参考： https://www.digitalocean.com/community/tutorials/how-to-add-and-delete-users-on-a-centos-7-server

```
$ sudo adduser cdp
$ sudo passwd cdp
$ sudo gpasswd -a cdp wheel
$ sudo lid -g wheel
```

这样的话会报错 no Privilege
然后查看/tmp/scm_prepare_node.XXXXX/scm_prepare_node.log

```
need root privileges but sudo requires password, exiting
```

然后修改/etc/sudoers.d下面的文件，添加

```
cdp ALL=(ALL) NOPASSWD:ALL
```

问题解决!

一些弯路：直接通过`visudo`来修改文件`/etc/sudoers`无法成功。原因未明

</p>
</details>

参考：

https://stackoverflow.com/questions/26428242/cloudera-manager-failed-to-authenticate-exhausted-available-authentication-met

```
# set passwd
$ sudo passwd root

# unlock
$ sudo passwd -u root
```

### sshd设置

sshd默认验证是的关着的，这里需要修改下

```ruby:set_sshd
$ sudo vi /etc/ssh/sshd_config
PermitRootLogin yes
PasswordAuthentication yes 
$ systemctl restart sshd.service
```

## swappiness
Inspect Cluster阶段出现警告，swap分区比重过大，要求下降（从60到10）
参考： https://qiita.com/ikuwow/items/f0b4d1f509a0b83b5d7e

```ruby:具体log
Cloudera recommends setting /proc/sys/vm/swappiness to a maximum of 10. Current setting is 30. Use the sysctl command to change this setting at run time and edit /etc/sysctl.conf for this setting to be saved after a reboot. You can continue with installation, but Cloudera Manager might report that your hosts are unhealthy because they are swapping. The following hosts are affected:
```

```ruby:一时修改
$ echo 10 > /proc/sys/vm/swappiness
$ cat /proc/sys/vm/swappiness
10
```

```ruby:不怕重启的修改
$ sudo vi /etc/sysctl.conf
vm.swappiness = 10
```


## transparent_hugepage
```ruby:具体log
Transparent Huge Page Compaction is enabled and can cause significant performance problems. Run "echo never > /sys/kernel/mm/transparent_hugepage/defrag" and "echo never > /sys/kernel/mm/transparent_hugepage/enabled" to disable this, and then add the same command to an init script such as /etc/rc.local so it will be set on system reboot. 
```

```ruby:照做
$ echo never > /sys/kernel/mm/transparent_hugepage/defrag
$ echo never > /sys/kernel/mm/transparent_hugepage/enabled
$ sudo vi /etc/rc.local
```

# 動作確認

## beeline

安装好的状态：zzeng01装着hive；没有任何密码和验证
尝试：从zzeng03登陆hive

1. CDP UI确认开启hiveserver2
1. 登陆zzeng03
1. beeline

```ruby
beeline -u jdbc:hive2://zzeng01.novalocal:10000
```
小插曲：用hive登陆的话，也会直接beeline登进去，CDP的优化？