EMRでRDSのMySQLを外部メタストアに指定するとTerminateされる #aws #emr


## 環境

* EMR: emr-4.3.0
* RDS: MySQL 5.6.27

## 現象

Configurationsに以下を指定してEMRを起動すると、`Terminated with errorsOn the master instance (i-xxxxxxxx), application provisioning failed` が発生してTerminateされる。

Configurations:

```
[
  {
    "classification": "hive-site",
    "properties": {
        "hive.exec.dynamic.partition.mode": "nonstrict",
        "javax.jdo.option.ConnectionURL": "jdbc:mysql://xxx.local:3306/hive",
        "javax.jdo.option.ConnectionDriverName": "com.mysql.jdbc.Driver",
        "javax.jdo.option.ConnectionUserName": "xxx",
        "javax.jdo.option.ConnectionPassword": "xxx"
    }
  }
]
```

マスターのプロビジョニング標準エラー出力に以下のエラーが出力される。

s3://aws-logs-xxxxxxxxxxxx-ap-northeast-1/elasticmapreduce/j-xxxxxxxxxxxxx/node/i-xxxxxxxx/provision-node/apps-phase/stderr.gz:

```
Error: /usr/lib/hive/bin/schematool -dbType mysql -initSchema -verbose returned 1 instead of one of [0]
Error: /Stage[main]/Hadoop_hive::Init_metastore_schema/Exec[init hive-metastore schema]/returns: change from notrun to 0 failed: /usr/lib/hive/bin/schematool -dbType mysql -initSchema -verbose returned 1 instead of one of [0]
Error: /Stage[main]/Hadoop_hive::Init_metastore_schema/Exec[init hive-metastore schema]: Failed to call refresh: /usr/lib/hive/bin/schematool -dbType mysql -initSchema -verbose returned 1 instead of one of [0]
Error: /Stage[main]/Hadoop_hive::Init_metastore_schema/Exec[init hive-metastore schema]: /usr/lib/hive/bin/schematool -dbType mysql -initSchema -verbose returned 1 instead of one of [0]
Warning: /Stage[main]/Hadoop_hive::Metastore_server/Service[hive-metastore]: Skipping because of failed dependencies
Warning: /Stage[main]/Hadoop_hive::Server/Package[hive-server2]: Skipping because of failed dependencies
Warning: /Stage[main]/Hadoop_hive::Server/Service[hive-server2]: Skipping because of failed dependencies
2016-04-16 02:16:54,022 ERROR main: Encountered a problem while provisioning
com.amazonaws.emr.node.provisioner.puppet.api.PuppetException: Unable to complete transaction and some changes were applied.
	at com.amazonaws.emr.node.provisioner.puppet.api.ApplyCommand.handleExitcode(ApplyCommand.java:74)
	at com.amazonaws.emr.node.provisioner.puppet.api.ApplyCommand.call(ApplyCommand.java:56)
	at com.amazonaws.emr.node.provisioner.bigtop.BigtopPuppeteer.applyPuppet(BigtopPuppeteer.java:49)
	at com.amazonaws.emr.node.provisioner.bigtop.BigtopDeployer.deploy(BigtopDeployer.java:21)
	at com.amazonaws.emr.node.provisioner.NodeProvisioner.provision(NodeProvisioner.java:28)
	at com.amazonaws.emr.node.provisioner.phase.PhaseWorkflow.work(PhaseWorkflow.java:51)
	at com.amazonaws.emr.node.provisioner.phase.ProvisionAppsPhase.perform(ProvisionAppsPhase.java:30)
	at com.amazonaws.emr.node.provisioner.Program.main(Program.java:17)
```

## 原因

`javax.jdo.option.ConnectionDriverName` は不要とのこと。

* [AWS Developer Forums: Unable to start EMR cluster with ...](https://forums.aws.amazon.com/thread.jspa?threadID=219887)

ドキュメントの例には `javax.jdo.option.ConnectionDriverName` を設定するようになっているので注意。

* [Create a Hive Metastore Outside the Cluster - Amazon Elastic MapReduce](http://docs.aws.amazon.com/ElasticMapReduce/latest/ReleaseGuide/emr-dev-create-metastore-outside.html)

## 対策

Configurationsから `javax.jdo.option.ConnectionDriverName` を削除してEMRを再構築。

```
[
  {
    "classification": "hive-site",
    "properties": {
        "hive.exec.dynamic.partition.mode": "nonstrict",
        "javax.jdo.option.ConnectionURL": "jdbc:mysql://xxx.local:3306/hive",
        "javax.jdo.option.ConnectionUserName": "xxx",
        "javax.jdo.option.ConnectionPassword": "xxx"
    }
  }
]
```