EC2でelasticsearch-curatorがDistributionNotFoundエラーを吐く場合の解決法

# Curator (elasticsearch-curator) とは

Elasticsearch社が公式に開発している、elasticsearchのIndexメンテナンスツールです。

+ [elasticsearch/curator](https://github.com/elasticsearch/curator)
+ 参考：[elasticsearch/curatorを使ってみる](http://qiita.com/ise_daisuke/items/266461ffc91246deb9ad)

# エラーと解決策

## Curator のインストール

```bash
$ python --version
Python 2.6.9

$ pip --version
pip 1.5.6 from /usr/lib/python2.6/site-packages (python 2.6)

# 2014/12時点での最新版は v2.0.2
# インストールまでは問題なく完了
$ sudo pip install elasticsearch-curator==2.0.2
```

## DistributionNotFoundエラー

インストールが終わり、動作確認しようとしたところでエラーが発生。

```bash
$ curator -v
Traceback (most recent call last):
  File "/usr/bin/curator", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: elasticsearch>=1.0.0,<2.0.0
```

エラーメッセージの内容的にはPythonの elasticsearch モジュールが見つからないという内容だが、elasticsearch-curator の依存モジュールとして一緒にインストールできている。

```bash
$ pip list | grep elasticsearch
elasticsearch (1.2.0)
elasticsearch-curator (2.0.2)
```

GithubやGoogle Groupに同様の問題が報告されているが、どれも明確な解決には至っていない様子...

+ https://github.com/elasticsearch/curator/issues/56
+ https://github.com/elasticsearch/curator/issues/77
+ https://groups.google.com/forum/#!msg/elasticsearch/wRR-7FAAvIo/yHCRrKo5gdYJ

実際、私もelasticsearchモジュールの再installなど試したがダメだった。

## 解決策

https://github.com/elasticsearch/curator/issues/56#issuecomment-51628636 で言及されている最後の手段として

> At worst, you'll have to run it from /Library/Python/2.7/site-packages/curator/curator.py (which is where pip installs it on mine).

を試すことに。

（https://github.com/elasticsearch/curator/issues/56#issuecomment-56980489 によるとv2系ではcurator.pyではなく、**curator_script.py**を使えとのこと。）

まずは **--dry-run** モードで

```bash
$ python /usr/lib/python2.6/site-packages/curator/curator_script.py -v
curator_script.py 2.0.2

# DRY RUN
$ python /usr/lib/python2.6/site-packages/curator/curator_script.py --host localhost --port 9200 --logfile /tmp/curator.log --loglevel DEBUG --dry-run delete --older-than 30

$ less /tmp/curator.log
2014-12-16 04:00:58,990 INFO      Job starting...
2014-12-16 04:00:58,990 INFO      DRY RUN MODE.  No changes will be made.
2014-12-16 04:00:59,187 DEBUG     Detected Elasticsearch version 1.4.1
2014-12-16 04:00:59,187 DEBUG     Setting default timestring for days to %Y.%m.%d
2014-12-16 04:00:59,187 DEBUG     Matching indices with pattern: logstash-%Y.%m.%d
2014-12-16 04:00:59,187 DEBUG     argdict = {'url_prefix': '', 'func': <function delete at 0x262c758>, 'prefix': 'logstash-', 'log_level': 'DEBUG', 'timestring': '%Y.%m.%d', 'dry_run': True, 'exclude_pattern': None, 'logformat': 'Default', 'auth': None, 'ssl': False, 'host': 'localhost', 'command': 'delete', 'time_unit': 'days', 'timeout': 30, 'debug': False, 'disk_space': None, 'log_file': '/tmp/curator.log', 'master_only': False, 'port': 9200, 'older_than': 30, 'suffix': ''}
2014-12-16 04:00:59,188 INFO      DRY RUN: Deleting indices...
2014-12-16 04:00:59,394 INFO      logstash-2014.12.11 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.12 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.13 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.14 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.15 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.16 is within the threshold period (30 days).
2014-12-16 04:00:59,421 INFO      DRY RUN: Speficied indices deleted.
2014-12-16 04:00:59,421 INFO      Done in 0:00:00.516716.
```

いけてるっぽいので実際のelasticsearchのIndexを対象に実行...

### bloom コマンド

```bash
$ python /usr/lib/python2.6/site-packages/curator/curator_script.py --host localhost --port 9200 --logfile /tmp/curator.log --loglevel DEBUG bloom --older-than 3

$ less /tmp/curator.log
2014-12-17 01:00:04,630 INFO      Job starting...
2014-12-17 01:00:04,646 DEBUG     Detected Elasticsearch version 1.4.1
2014-12-17 01:00:04,646 DEBUG     Setting default timestring for days to %Y.%m.%d
2014-12-17 01:00:04,646 DEBUG     Matching indices with pattern: logstash-%Y.%m.%d
2014-12-17 01:00:04,646 DEBUG     argdict = {'url_prefix': '', 'prefix': 'logstash-', 'log_level': 'DEBUG', 'timestring': '%Y.%m.%d', 'dry_run': False, 'exclude_pattern': None, 'logformat': 'Default', 'auth': None, 'ssl': False, 'host': 'localhost', 'command': 'bloom', 'time_unit': 'days', 'timeout': 30, 'debug': False, 'func': <function bloom at 0xc49668>, 'log_file': '/tmp/curator.log', 'master_only': False, 'port': 9200, 'older_than': 3, 'suffix': ''}
2014-12-17 01:00:04,647 INFO      Disabling the bloom filter cache for indices...
2014-12-17 01:00:04,895 INFO      disable_bloom_filter operation succeeded on logstash-2014.12.11
2014-12-17 01:00:05,106 INFO      disable_bloom_filter operation succeeded on logstash-2014.12.12
2014-12-17 01:00:05,428 INFO      disable_bloom_filter operation succeeded on logstash-2014.12.13
2014-12-17 01:00:05,888 INFO      disable_bloom_filter operation succeeded on logstash-2014.12.14
2014-12-17 01:00:05,888 INFO      logstash-2014.12.15 is within the threshold period (3 days).
2014-12-17 01:00:05,888 INFO      logstash-2014.12.16 is within the threshold period (3 days).
2014-12-17 01:00:05,888 INFO      logstash-2014.12.17 is within the threshold period (3 days).
2014-12-17 01:00:05,888 INFO      Disabled bloom filter cache for specified indices.
2014-12-17 01:00:05,889 INFO      Done in 0:00:01.415160.
```

### close コマンド

```bash
$ python /usr/lib/python2.6/site-packages/curator/curator_script.py --host localhost --port 9200 --logfile /tmp/curator.log --loglevel DEBUG close --older-than 4

$ less /tmp/curator.log
2014-12-17 01:00:06,946 INFO      Job starting...
2014-12-17 01:00:07,024 DEBUG     Detected Elasticsearch version 1.4.1
2014-12-17 01:00:07,024 DEBUG     Setting default timestring for days to %Y.%m.%d
2014-12-17 01:00:07,024 DEBUG     Matching indices with pattern: logstash-%Y.%m.%d
2014-12-17 01:00:07,025 DEBUG     argdict = {'url_prefix': '', 'prefix': 'logstash-', 'log_level': 'DEBUG', 'timestring': '%Y.%m.%d', 'dry_run': False, 'exclude_pattern': None, 'logformat': 'Default', 'auth': None, 'ssl': False, 'host': 'localhost', 'command': 'close', 'time_unit': 'days', 'timeout': 30, 'debug': False, 'func': <function close at 0x2b0c6e0>, 'log_file': '/tmp/curator.log', 'master_only': False, 'port': 9200, 'older_than': 4, 'suffix': ''}
2014-12-17 01:00:07,025 INFO      Closing indices...
2014-12-17 01:00:08,457 INFO      close_index operation succeeded on logstash-2014.12.11
2014-12-17 01:00:08,841 INFO      close_index operation succeeded on logstash-2014.12.12
2014-12-17 01:00:09,225 INFO      close_index operation succeeded on logstash-2014.12.13
2014-12-17 01:00:09,225 INFO      logstash-2014.12.14 is within the threshold period (4 days).
2014-12-17 01:00:09,226 INFO      logstash-2014.12.15 is within the threshold period (4 days).
2014-12-17 01:00:09,226 INFO      logstash-2014.12.16 is within the threshold period (4 days).
2014-12-17 01:00:09,226 INFO      logstash-2014.12.17 is within the threshold period (4 days).
2014-12-17 01:00:09,226 INFO      Closed specified indices.
2014-12-17 01:00:09,226 INFO      Done in 0:00:02.293359.
```

### delete コマンド

```bash
$ python /usr/lib/python2.6/site-packages/curator/curator_script.py --host localhost --port 9200 --logfile /tmp/curator.log --loglevel DEBUG delete --older-than 5

$ less /tmp/curator.log
2014-12-17 01:00:10,375 INFO      Job starting...
2014-12-17 01:00:10,382 DEBUG     Detected Elasticsearch version 1.4.1
2014-12-17 01:00:10,382 DEBUG     Setting default timestring for days to %Y.%m.%d
2014-12-17 01:00:10,382 DEBUG     Matching indices with pattern: logstash-%Y.%m.%d
2014-12-17 01:00:10,382 DEBUG     argdict = {'url_prefix': '', 'func': <function delete at 0x2578758>, 'prefix': 'logstash-', 'log_level': 'DEBUG', 'timestring': '%Y.%m.%d', 'dry_run': False, 'exclude_pattern': None, 'logformat': 'Default', 'auth': None, 'ssl': False, 'host': 'localhost', 'command': 'delete', 'time_unit': 'days', 'timeout': 30, 'debug': False, 'disk_space': None, 'log_file': '/tmp/curator.log', 'master_only': False, 'port': 9200, 'older_than': 5, 'suffix': ''}
2014-12-17 01:00:10,382 INFO      Deleting indices...
2014-12-17 01:00:11,632 INFO      delete_index operation succeeded on logstash-2014.12.11
2014-12-17 01:00:11,727 INFO      delete_index operation succeeded on logstash-2014.12.12
2014-12-17 01:00:11,727 INFO      logstash-2014.12.13 is within the threshold period (5 days).
2014-12-17 01:00:11,727 INFO      logstash-2014.12.14 is within the threshold period (5 days).
2014-12-17 01:00:11,727 INFO      logstash-2014.12.15 is within the threshold period (5 days).
2014-12-17 01:00:11,728 INFO      logstash-2014.12.16 is within the threshold period (5 days).
2014-12-17 01:00:11,728 INFO      logstash-2014.12.17 is within the threshold period (5 days).
2014-12-17 01:00:11,728 INFO      Speficied indices deleted.
2014-12-17 01:00:11,728 INFO      Done in 0:00:01.437909.
```

できたー！٩(๑´3｀๑)۶

## おまけ

aliasに設定し、Pythonのsite-packagesの場所も動的に決まるようにしておくとお後がよろしい

```bash
alias curator="python "`python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"`/curator/curator_script.py
```