EC2上にElasticsearch環境構築

# 概要
AWS EC2インスタンス上にElasticsearchを導入します。
Elasticsearchのバージョンは `8.5.3`です。



# 参考文献
Elasticsearch 公式ガイド : https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
Logstash 公式リファレンス : https://www.elastic.co/guide/en/logstash/current/index.html
Kibana 公式ガイド : https://www.elastic.co/guide/en/kibana/current/index.html


# 目次
[1. EC2インスタンスSSH接続](#1-ec2インスタンス-ssh接続)
[2. Amazon Correttoのインストール](#2-amazon-correttoのインストール)
[3. Logstashのインストール](#3-logstashのインストール)
[4. Elasticsearchのインストール](#4-elasticsearchのインストール)
[5. Kibanaのインストール](#5-kibanaのインストール)
[6. ブラウザ確認](#6-ブラウザ確認)


# 1. EC2インスタンス SSH接続
AWS EC2インスタンスにTera Termを使ってSSH接続を行います。

Tera Termを立ち上げてから
ホスト(T)にEC2インスタンスのIPアドレスを指定します。

<img width="700" alt="teraterm1.png" src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/711148/b7d38f5c-c610-24e8-31a4-cf025ba6c464.png">

続行(C)をクリック
<img width="700" alt="teraterm2.png" src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/711148/3b9f6ebd-d6dc-5206-7014-69b2c23f4236.png">

ユーザ名(N)に`ec2-user`を記載
`RSA/DSA/ECDSA/ED25519鍵を使う`にチェックを入れて
秘密鍵(K)に画面右の`...`ボタンから秘密鍵を選択
設定したらOKを押す
<img width="700" alt="teraterm3.png" src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/711148/30c0d616-fbb9-b359-605c-f469edd1fd6d.png">

EC2インスタンス SSH接続完了
<img width="700" alt="teraterm4.png" src="https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/711148/3e7bae90-67cb-d331-f8e8-4d81be9a5709.png">







# 2. Amazon Correttoのインストール
Elasticsearch,logstashでJava17が必要
今回はOpenJDK ディストリビューションであるAmazon Correttoの17をインストールして使う。

特権ユーザーに変更して以下インストール作業実行する

```
sudo su -
```


Amazon Corretto 17のインストール

``` bash
yum install java-17-amazon-corretto
```

インストール(version)確認

```
java -version
```

レスポンスで下記バージョン情報が出ればOK

```
openjdk version "17.0.2" 2022-01-18 LTS
OpenJDK Runtime Environment Corretto-17.0.2.8.1 (build 17.0.2+8-LTS)
OpenJDK 64-Bit Server VM Corretto-17.0.2.8.1 (build 17.0.2+8-LTS, mixed mode, sharing)
```
</br>
</br>

# 3. Logstashのインストール
参考URL：https://www.elastic.co/guide/en/logstash/current/installing-logstash.html


ElasticGPGキーのインポート
(標準出力には何も出ないが問題なし)

```bash
sudo rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
```

/etc/yum.repos.d/配下に「**logstash.repo**」ファイルとして
以下の内容で.repoファイルを作成する。
※enabled=1でリポジトリを有効化(任意)
　install時にレポジトリの指定が不要

```
vi /etc/yum.repos.d/logstash.repo
```

```/etc/yum.repos.d/logstash.repo
[logstash-8.x]
name=Elastic repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
```

Logstashのインストール
version 指定する場合(ver 8.1.2)

```bash
yum install -y logstash-8.1.2
```
最新versionでLogstashをinstall する場合

```bash
yum install logstash
```

Logstash 起動と自動起動設定

```bash
systemctl daemon-reload
```

```
systemctl enable logstash
```

レスポンス　

```
Created symlink from /etc/systemd/system/multi-user.target.wants/logstash.service to /etc/systemd/system/logstash.service.
```


</br>
</br>

```
systemctl start logstash
```

```
systemctl status logstash
```




status確認 レスポンス
Active項目が<font color="OliveDrab">active (running)</font>になっていればOK

```
● logstash.service - logstash
   Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2021-12-17 01:35:57 UTC; 1min 19s ago
```



</br>

# 4. Elasticsearchのインストール
参考URL：https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html



/etc/yum.repos.d/配下に「**elasticsearch.repo**」ファイルとして
以下の内容で.repoファイルを作成する。

```
vi /etc/yum.repos.d/elasticsearch.repo
```

※enabled=0でリポジトリを無効(任意)
　install時にレポジトリの指定が必要

```/etc/yum.repos.d/elasticsearch.repo
[elasticsearch]
name=Elasticsearch repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=0
autorefresh=1
type=rpm-md
```

Elasticsearchのインストール

```bash
sudo yum install --enablerepo=elasticsearch elasticsearch
```

レスポンス(一部)
version8.0からyumでinstallした際に下記のように
elastic(スーパーユーザー)の初期のパスワードが設定されている。



:::note warn
Vol.3でセキュリティを設定する際に初期パスワード使用するので必ずメモしておく!!

**built-in superuser is : XXXXXXXXX**  　＜＜ーーーーーー  初期パスワード
:::



```bash
--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : Bn6r0mHRu23fBxeuvuVL <------初期パスワード!!

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token <token-here>'
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with
 '/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

```


jvm.optionsの編集

```bash
vi /etc/elasticsearch/jvm.options
```


ヒープサイズのコメントの後に2行追加

```bash:/etc/elasticsearch/jvm.options
################################################################
## IMPORTANT: JVM heap size
################################################################
##
## The heap size is automatically configured by Elasticsearch
## based on the available memory in your system and the roles
## each node is configured to fulfill. If specifying heap is
## required, it should be done through a file in jvm.options.d,
## and the min and max should be set to the same value. For
## example, to set the heap to 4 GB, create a new file in the
## jvm.options.d directory containing these lines:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/7.15/heap-size.html
## for more information
##
################################################################

-Xms1g
-Xmx1g
```


elasticsearch.yml設定
`discovery.type`パラメータを設定します。

```bash
vi /etc/elasticsearch/elasticsearch.yml
```


```/etc/elasticsearch/elasticsearch.yml
#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 01-01-2023 13:37:00
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
#cluster.initial_master_nodes: ["ip-172-31-21-144.ap-northeast-1.compute.internal"]
discovery.type: single-node　　　　<------------◆◆追記◆◆

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------
```



Elasticsearch起動と自動起動設定

```bash
systemctl daemon-reload
```
```bash
systemctl enable elasticsearch
```
```bash
systemctl start elasticsearch
```
```bash
systemctl status elasticsearch
```



</br>
</br>

# 5. Kibanaのインストール
参考URL：https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html

/etc/yum.repos.d/配下に「**kibana.repo**」ファイルとして
以下の内容で.repoファイルを作成する。

```
vi /etc/yum.repos.d/kibana.repo
```

※enabled=1でリポジトリを有効化(任意)
　install時にレポジトリの指定が不要

```/etc/yum.repos.d/kibana.repo
[kibana-8.x]
name=Kibana repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
```



Kibanaのインストール

```bash
yum install kibana
```


kibana.ymlの編集

```bash
vi /etc/kibana/kibana.yml
```

```/etc/kibana/kibana.yml
# For more configuration options see the configuration guide for Kibana in
# https://www.elastic.co/guide/index.html

# =================== System: Kibana Server ===================
# Kibana is served by a back end server. This setting specifies the port to use.
server.port: 5601

# Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
# The default is 'localhost', which usually means remote machines will not be able to connect.
# To allow connections from remote users, set this parameter to a non-loopback address.
server.host: "0.0.0.0"

# Enables you to specify a path to mount Kibana at if you are running behind a proxy.
# Use the `server.rewriteBasePath` setting to tell Kibana if it should remove the basePath
# from requests it receives, and to prevent a deprecation warning at startup.
# This setting cannot end in a slash.
#server.basePath: ""

# Specifies whether Kibana should rewrite requests that are prefixed with
# `server.basePath` or require that they are rewritten by your reverse proxy.
# Defaults to `false`.
#server.rewriteBasePath: false

# Specifies the public URL at which Kibana is available for end users. If
# `server.basePath` is configured this URL should end with the same basePath.
server.publicBaseUrl: "http://localhost:5601"

# The maximum payload size in bytes for incoming server requests.
#server.maxPayload: 1048576

# The Kibana server's name. This is used for display purposes.
#server.name: "your-hostname"

# =================== System: Kibana Server (Optional) ===================
# Enables SSL and paths to the PEM-format SSL certificate and SSL key files, respectively.
# These settings enable SSL for outgoing requests from the Kibana server to the browser.
#server.ssl.enabled: false
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# =================== System: Elasticsearch ===================
# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["http://localhost:9200"]
```

kibana.yml修正箇所まとめ

```/etc/kibana/kibana.yml
server.port: 5601

server.host: "0.0.0.0"

server.publicBaseUrl: "http://localhost:5601"

elasticsearch.hosts: ["http://localhost:9200"]
```

Kibana 起動と自動起動設定

```
systemctl daemon-reload
```
```
systemctl enable kibana
```
```
systemctl start kibana
```
```
systemctl status kibana
```



# 6. ブラウザ確認
ブラウザで [http://&lt;TeraTermにログインしたIP Adrress&gt;:5601](http://localhost:5601) でKibanaにアクセスします。
アクセスするとEnrollment tokenが求められます。
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/711148/0b1aa78f-6ba1-0c9f-f057-0ded4dd57b68.png)


***