pipでmaec 4.0.1.0をインストールしようとするとエラーが出る


Cuckoo Sandboxをセットアップしようとして`pip`でmaecのバージョン`4.0.1.0`をインストールしようとしたらエラーが出たのでその調査。
試行錯誤した結果なので、他に正しい方法が絶対あると思うので誰か教えてくださいませ。
（※追記しました。）

#環境
- OS
 - Ubuntu 14.04
- maec
 - 4.0.1.0

#エラー内容
```shell-session
$ sudo pip install maec==4.0.1.0
Downloading/unpacking maec==4.0.1.0
  Downloading maec-4.0.1.0.tar.gz (114kB): 114kB downloaded
  Running setup.py (path:/tmp/pip_build_root/maec/setup.py) egg_info for package maec
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_root/maec/setup.py", line 1, in <module>
        import maec
      File "maec/__init__.py", line 6, in <module>
        import bindings.maec_bundle as bundle_binding
      File "maec/bindings/maec_bundle.py", line 12, in <module>
        from cybox.bindings import cybox_core
    ImportError: No module named cybox.bindings
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_root/maec/setup.py", line 1, in <module>

    import maec

  File "maec/__init__.py", line 6, in <module>

    import bindings.maec_bundle as bundle_binding

  File "maec/bindings/maec_bundle.py", line 12, in <module>

    from cybox.bindings import cybox_core

ImportError: No module named cybox.bindings

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/maec
Storing debug log for failure in /home/vagrant/.pip/pip.log
```

こんな感じのエラーが出ます。

##cyboxのインストール
エラー内容を見るとcyboxがないと怒られているようなので、cyboxを入れます。

```console
$ sudo pip install cybox

（中略）

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/src

creating build/temp.linux-x86_64-2.7/src/lxml

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w

x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 4

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-unrw2N-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
Storing debug log for failure in /home/vagrant/.pip/pip.log
```

エラーが出ます。次から次へとエラーが出て心が折れそうになりますが、公式のドキュメントを見てみます。
http://cybox.readthedocs.org/en/latest/installation.html
いくつか依存しているライブラリがあるようです。
それを入れれば解決しそうです。

```console
$ sudo apt-get install libxml2-dev libxslt1-dev zlib1g-dev
```

そして再びインストール。

```console
$ sudo pip install cybox

（中略）

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/src

creating build/temp.linux-x86_64-2.7/src/lxml

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w

x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 4

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-unrw2N-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
Storing debug log for failure in /home/vagrant/.pip/pip.log
```

エラー変わってないじゃん！
どうやら`lxml`がうまく行ってなさそう、とあたりを付けて今度は`lxml`のインストールを試してみます。

##lxmlのインストール
`pip`でインストールできそうなのでやってみます。

```console
$ sudo pip install lxml
（中略）
creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/src

creating build/temp.linux-x86_64-2.7/src/lxml

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w

x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 4

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-7X8_bC-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
Storing debug log for failure in /home/vagrant/.pip/pip.log
```

なんか同じっぽいエラーが出るので、やはりlxmlが原因だったぽいです。
一応公式ドキュメント（http://lxml.de/installation.html） を見ていると、ビルドを早めるためにCの最適化をオフにするやり方があったので、念のため試してみると。。

```console
vagrant@vagrant-ubuntu-trusty-64:~$ sudo CFLAGS="-O0" pip install lxml
Downloading/unpacking lxml
  Downloading lxml-3.4.0.tar.gz (3.5MB): 3.5MB downloaded
  Running setup.py (path:/tmp/pip_build_root/lxml/setup.py) egg_info for package lxml
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.4.0.
    Building without Cython.
    Using build configuration of libxslt 1.1.28

    warning: no previously-included files found matching '*.py'
Installing collected packages: lxml
  Running setup.py install for lxml
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.4.0.
    Building without Cython.
    Using build configuration of libxslt 1.1.28
    building 'lxml.etree' extension
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -O0 -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
    x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -O0 build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/etree.so
    building 'lxml.objectify' extension
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -O0 -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.objectify.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -w
    x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -O0 build/temp.linux-x86_64-2.7/src/lxml/lxml.objectify.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/objectify.so

Successfully installed lxml
Cleaning up...
```

いけたっぽい。。
最適化するところでコケていたのだろうか。
詳細を調べたいのですが、疲れたので誰かお願いします。

##cyboxのインストール（再チャレンジ）
```console
$ sudo pip install cybox
Requirement already satisfied (use --upgrade to upgrade): cybox in /usr/local/lib/python2.7/dist-packages
Requirement already satisfied (use --upgrade to upgrade): lxml>=2.3 in /usr/local/lib/python2.7/dist-packages (from cybox)
Downloading/unpacking python-dateutil (from cybox)
  Downloading python-dateutil-2.2.tar.gz (259kB): 259kB downloaded
  Running setup.py (path:/tmp/pip_build_root/python-dateutil/setup.py) egg_info for package python-dateutil

Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/dist-packages (from python-dateutil->cybox)
Installing collected packages: python-dateutil
  Running setup.py install for python-dateutil

Successfully installed python-dateutil
Cleaning up...
```

やっといけた。

##maecのインストール（再チャレンジ）
これでいけるはず！

```console
$ sudo pip install maec==4.0.1.0
Downloading/unpacking maec==4.0.1.0
  Downloading maec-4.0.1.0.tar.gz (114kB): 114kB downloaded
  Running setup.py (path:/tmp/pip_build_root/maec/setup.py) egg_info for package maec
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_root/maec/setup.py", line 1, in <module>
        import maec
      File "maec/__init__.py", line 6, in <module>
        import bindings.maec_bundle as bundle_binding
      File "maec/bindings/maec_bundle.py", line 5424, in <module>
        'Relationships': cybox_core.RelationshipsType,
    AttributeError: 'module' object has no attribute 'RelationshipsType'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_root/maec/setup.py", line 1, in <module>

    import maec

  File "maec/__init__.py", line 6, in <module>

    import bindings.maec_bundle as bundle_binding

  File "maec/bindings/maec_bundle.py", line 5424, in <module>

    'Relationships': cybox_core.RelationshipsType,

AttributeError: 'module' object has no attribute 'RelationshipsType'

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/maec
Storing debug log for failure in /home/vagrant/.pip/pip.log
```

違うエラー。。もう無理だ。。
自分を励まし何とか調べてみたところ、どうやら`cybox`のバージョン違いのようです。
v2.0.1.4を落としてきたらいけるとのこと。

##cyboxのインストール（再々チャレンジ）
```shell-session
$ wget https://github.com/CybOXProject/python-cybox/archive/v2.0.1.4.tar.gz
$ tar zxvf v2.0.1.4.tar.gz
$ cd python-cybox-2.0.1.4/
$ sudo python setup.py install
（中略）
Installed /usr/local/lib/python2.7/dist-packages/cybox-2.0.1.4-py2.7.egg
Processing dependencies for cybox==2.0.1.4
Searching for python-dateutil==2.2
Best match: python-dateutil 2.2
Processing python_dateutil-2.2-py2.7.egg
python-dateutil 2.2 is already the active version in easy-install.pth

Using /usr/local/lib/python2.7/dist-packages/python_dateutil-2.2-py2.7.egg
Searching for lxml==3.4.0
Best match: lxml 3.4.0
Processing lxml-3.4.0-py2.7-linux-x86_64.egg
lxml 3.4.0 is already the active version in easy-install.pth

Using /usr/local/lib/python2.7/dist-packages/lxml-3.4.0-py2.7-linux-x86_64.egg
Searching for six==1.5.2
Best match: six 1.5.2
six 1.5.2 is already the active version in easy-install.pth

Using /usr/lib/python2.7/dist-packages
Finished processing dependencies for cybox==2.0.1.4
```

##maecのインストール（再々チャレンジ）
```console
$ sudo pip install maec==4.0.1.0
Downloading/unpacking maec==4.0.1.0
  Downloading maec-4.0.1.0.tar.gz (114kB): 114kB downloaded
  Running setup.py (path:/tmp/pip_build_root/maec/setup.py) egg_info for package maec

Requirement already satisfied (use --upgrade to upgrade): lxml>=2.3 in /usr/local/lib/python2.7/dist-packages/lxml-3.4.0-py2.7-linux-x86_64.egg (from maec==4.0.1.0)
Requirement already satisfied (use --upgrade to upgrade): cybox>=2.0.1.0,<2.0.2.0 in /usr/local/lib/python2.7/dist-packages/cybox-2.0.1.4-py2.7.egg (from maec==4.0.1.0)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil in /usr/local/lib/python2.7/dist-packages/python_dateutil-2.2-py2.7.egg (from cybox>=2.0.1.0,<2.0.2.0->maec==4.0.1.0)
Requirement already satisfied (use --upgrade to upgrade): six in /usr/lib/python2.7/dist-packages (from python-dateutil->cybox>=2.0.1.0,<2.0.2.0->maec==4.0.1.0)
Installing collected packages: maec
  Running setup.py install for maec

Successfully installed maec
Cleaning up...
```

おぉ。。ついにインストールできた。。
ということでようやく出来ました！

#まとめ
試行錯誤の過程を長々と書きましたが、結局まとめると以下の順番です。

1. lxmlを入れる
2. cybox(v2.0.1.4)を入れる
3. maec(4.0.1.0)を入れる


```shell-session
#lxmlが依存するライブラリを入れる
$ sudo apt-get install libxml2-dev libxslt1-dev zlib1g-dev
#lxmlを入れる
$ sudo CFLAGS="-O0" pip install lxml
#cyboxを入れる
$ wget https://github.com/CybOXProject/python-cybox/archive/v2.0.1.4.tar.gz
$ tar zxvf v2.0.1.4.tar.gz
$ cd python-cybox-2.0.1.4/
$ sudo python setup.py install
#maecを入れる
$ sudo pip install maec==4.0.1.0
```

#追記
lxmlはapt-getで入れられるようです。

```console
$ sudo apt-get install python-lxml
```

これを用いて、上のコマンドを修正すると、

```shell-session
#python-lxmlだけでいいかも
$ sudo apt-get install libxml2-dev libxslt1-dev zlib1g-dev python-lxml
#cyboxを入れる
$ wget https://github.com/CybOXProject/python-cybox/archive/v2.0.1.4.tar.gz
$ tar zxvf v2.0.1.4.tar.gz
$ cd python-cybox-2.0.1.4/
$ sudo python setup.py install
#maecを入れる
$ sudo pip install maec==4.0.1.0
```

大分簡略化されました。