OpenMDAO/OpenMDAO-Framework — Issue #696: Installation from local "dist" mirror fails
Author: nathanjsmith
Created: 2011-11-11 18:36:22+00:00
State: open

# Problem statement

go-openmdao-dev.py normally attempts to download all of its dependencies from `openmdao.org/dist/index.html` referenced sources. For machines on which this site is not accessible (e.g. due to firewall), it is necessary to install from a local repository. This is partially supported by the script, but in practice there are issues that prevent it from working all the way:

# Specific issues

1. go-openmdao.py, for the most part, ignores the --disturl argument. Internally, there is a variable `url` that is explicitly set to `http://openmdao.org/dists`.

2. Traits requires `setupdocs>=1.0`. `SetupDocs-1.0.5.tar.gz` is available on OpenMDAO's repository. However, the Traits installer attempts to install SetupDocs automatically, but does not know the value of the `--disturl` flag so it tries to download it from `http://pypi.python.org/...`.

# Potential solutions:

1. If go-openmdao.py had `url` set to `openmdao_url` instead of explicitly assigned 'http://openmdao.org/dists`, then the `--disturl` command line argument would work to control the source of installable packages.

2.The --disturl flag should be passed down to packages being installed by OpenMDAO's installer script so that they know to install from a local repository rather than the Internet.
* A workaround is to make OpenMDAO depend on `setupdocs>=1.0` and to place this dependency before the dependency on `Traits>=3.3.0`. This forces OpenMDAO to satisfy Traits' dependency before the Traits installer runs.

# How this was tested:

I installed CentOS-5.0 (yes, it is *very* old) in a 64-bit virtual machine. I then downloaded, compiled and installed Python 2.6.7 and zlib-1.2.5 to a local directory:

INSTALL_ROOT=${PWD}
export PATH=${INSTALL_ROOT}/openmdao/bin:${PATH}
export LD_LIBRARY_PATH=${INSTALL_ROOT}/openmdao/lib:${LD_LIBRARY_PATH}
export CFLAGS="-I${INSTALL_ROOT}/openmdao -I${INSTALL_ROOT}/openmdao/include"
export LDFLAGS="-L${INSTALL_ROOT}/openmdao -L${INSTALL_ROOT}/openmdao/lib"

cd zlib-1.2.5
./configure --prefix=${INSTALL_ROOT}/openmdao
make
make install
cd ../

cd Python-2.6.7
./configure --prefix=${INSTALL_ROOT}/openmdao
make
make install
cd ../

I then mirrored OpenMDAO's distribution repository:

mkdir dist
cd dist
wget --recursive --no-clobber --page-requisites --html-extension --convert-links --domains openmdao.org --no-parent http://openmdao.org/dists/index.html
wget http://pypi.python.org/packages/source/p/pip/pip-1.0.2.tar.gz
cd -

I then ran OpenMDAO's installer:

python go-openmdao-dev.py --extra-search-dirs=${INSTALL_ROOT}/dist/openmdao.org/dists --extra-search-dir=dist --disturl=${INSTALL_ROOT}/dist/openmdao.org/dists --noprereqs --nogui

Without the above suggested changes, the installer failed in various ways.
* Without setting `url = disturl`, the installer would attempt to download the packages from `http://openmdao.org/dist`.
* Without adding `setupdocs>=1.0` to OpenMDAO's requirements, Traits' installer would attempt to download SetupDocs-1.0.5 from Enthought's dist site.