Repository: conda/conda-build

Issue:
conda-build either stops or endless cycle of "updating index in:"
```
update: a completely fresh install of anaconda3 does not solve this problem...


conda build stops at indexing, increasing timeout does not help.

bjorn@bjorn-ThinkPad-T450s:~/python_packages/pydna$ conda build .
conda: command not found
bjorn@bjorn-ThinkPad-T450s:~/python_packages/pydna$ bjorn3
(bjorn3) bjorn@bjorn-ThinkPad-T450s:~/python_packages/pydna$ conda build .
Cloning into '/home/bjorn/anaconda/conda-bld/pydna_1475659341958/work'...
done.
checkout: 'HEAD'
Your branch is up-to-date with 'origin/_conda_cache_origin_head'.
==> git log -n1 <==

commit 86fa9f37cb11bc8d7902233001f24764e8c43870
Author: BjornFJohansson <bjorn_johansson@bio.uminho.pt>
Date:   Thu Sep 15 11:42:44 2016 +0100

    changed back source in meta.yaml


==> git describe --tags --dirty <==

1.0.2


==> git status <==

On branch _conda_cache_origin_head
Your branch is up-to-date with 'origin/_conda_cache_origin_head'.
nothing to commit, working tree clean


BUILD START: pydna-1.0.2-py35_0
updating index in: /home/bjorn/anaconda/conda-bld/linux-64
(stops here)...

tried to remove pkgs folder, but now conda build stops with an endless cycle of output ending in 
"updating index in: /home/bjorn/anaconda/conda-bld/noarch" see below.

In conda-bld i get a new folder with each cycle named like this:

/home/bjorn/anaconda/conda-bld/pydna_1475657706204
/home/bjorn/anaconda/conda-bld/pydna_1475657701788
/home/bjorn/anaconda/conda-bld/pydna_1475657697078
/home/bjorn/anaconda/conda-bld/pydna_1475657692475
/home/bjorn/anaconda/conda-bld/pydna_1475657687994
/home/bjorn/anaconda/conda-bld/pydna_1475657683599
... ()

The only thing that appears in 
/home/bjorn/anaconda/conda-bld/linux-64/ are these two files:

/home/bjorn/anaconda/conda-bld/linux-64/repodata.json
/home/bjorn/anaconda/conda-bld/linux-64/repodata.json.bz2

This issuee is probably connected to [this post](https://groups.google.com/a/continuum.io/d/msg/anaconda/xNp9oMi2vQE/0khuq_A8AgAJ), 
and maybe this [issue](https://github.com/conda/conda-build/issues/1339).

The same thing seem to happen on an attempted build using [drone.io](https://drone.io/github.com/BjornFJohansson/pydna/latest).

I tried to increase the timeout to 900 seconds here: 

(file is /conda_build/config.py)

        Setting = namedtuple("ConfigSetting", "name, default")
        values = [Setting('activate', True),
                  Setting('anaconda_upload', binstar_upload),
                  Setting('channel_urls', ()),
                  Setting('dirty', False),
                  Setting('include_recipe', True),
                  Setting('keep_old_work', False),
                  Setting('noarch', False),
                  Setting('no_download_source', False),
                  Setting('override_channels', False),
                  Setting('skip_existing', False),
                  Setting('token', None),
                  Setting('user', None),
                  Setting('verbose', False),
                  Setting('debug', False),
                  Setting('timeout', 900),
                  Setting('subdir', subdir),
                  Setting('bits', bits),
                  Setting('platform', platform),
                  Setting('set_build_id', True),
                  Setting('disable_pip', False)
                  ] 




## Endless cycle...

(bjorn3) bjorn@bjorn-ThinkPad-T450s:~/python_packages/pydna$ git --version
git version 2.10.1
(bjorn3) bjorn@bjorn-ThinkPad-T450s:~/python_packages/pydna$ git tag
0.8.0
0.8.1
0.8.2
0.8.3
0.8.4
0.9.0
0.9.1
0.9.2
0.9.3
0.9.4
0.9.5
0.9.6
0.9.7
0.9.8
0.9.9
1.0.0
1.0.1
1.0.2
(bjorn3) bjorn@bjorn-ThinkPad-T450s:~/python_packages/pydna$ conda build .
Cloning into bare repository '/home/bjorn/anaconda/conda-bld/git_cache/home/bjorn/python_packages/pydna'...
done.
Cloning into '/home/bjorn/anaconda/conda-bld/pydna_1475657602868/work'...
done.
checkout: 'HEAD'
Your branch is up-to-date with 'origin/py3'.
==> git log -n1 <==

commit 86fa9f37cb11bc8d7902233001f24764e8c43870
Author: BjornFJohansson <bjorn_johansson@bio.uminho.pt>
Date:   Thu Sep 15 11:42:44 2016 +0100

    changed back source in meta.yaml


==> git describe --tags --dirty <==

1.0.2


==> git status <==

On branch py3
Your branch is up-to-date with 'origin/py3'.
nothing to commit, working tree clean


BUILD START: pydna-1.0.2-py35_0
updating index in: /home/bjorn/anaconda/conda-bld/linux-64
updating index in: /home/bjorn/anaconda/conda-bld/noarch
...# If you want to upload this package to anaconda.org later, type:
#
# $ anaconda upload /home/bjorn/anaconda/conda-bld/linux-64/pydna-1.0.2-py35_0.tar.bz2
#
# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

From /home/bjorn/python_packages/pydna
 * [new ref]                    -> _conda_cache_origin_head
Cloning into '/home/bjorn/anaconda/conda-bld/pydna_1475657608447/work'...
done.
checkout: 'HEAD'
Your branch is up-to-date with 'origin/_conda_cache_origin_head'.
==> git log -n1 <==

commit 86fa9f37cb11bc8d7902233001f24764e8c43870
Author: BjornFJohansson <bjorn_johansson@bio.uminho.pt>
Date:   Thu Sep 15 11:42:44 2016 +0100

    changed back source in meta.yaml


==> git describe --tags --dirty <==

1.0.2


==> git status <==

On branch _conda_cache_origin_head
Your branch is up-to-date with 'origin/_conda_cache_origin_head'.
nothing to commit, working tree clean


BUILD START: pydna-1.0.2-py35_0
updating index in: /home/bjorn/anaconda/conda-bld/linux-64
updating index in: /home/bjorn/anaconda/conda-bld/noarch
...# If you want to upload this package to anaconda.org later, type:
#
# $ anaconda upload /home/bjorn/anaconda/conda-bld/linux-64/pydna-1.0.2-py35_0.tar.bz2
#
# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

Cloning into '/home/bjorn/anaconda/conda-bld/pydna_1475657612191/work'...
done.
checkout: 'HEAD'
Your branch is up-to-date with 'origin/_conda_cache_origin_head'.
==> git log -n1 <==

commit 86fa9f37cb11bc8d7902233001f24764e8c43870
Author: BjornFJohansson <bjorn_johansson@bio.uminho.pt>
Date:   Thu Sep 15 11:42:44 2016 +0100

    changed back source in meta.yaml


==> git describe --tags --dirty <==

1.0.2


==> git status <==

On branch _conda_cache_origin_head
Your branch is up-to-date with 'origin/_conda_cache_origin_head'.
nothing to commit, working tree clean


BUILD START: pydna-1.0.2-py35_0
updating index in: /home/bjorn/anaconda/conda-bld/linux-64
updating index in: /home/bjorn/anaconda/conda-bld/noarch
...# If you want to upload this package to anaconda.org later, type:
#
# $ anaconda upload /home/bjorn/anaconda/conda-bld/linux-64/pydna-1.0.2-py35_0.tar.bz2
#
# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

Cloning into '/home/bjorn/anaconda/conda-bld/pydna_1475657616926/work'...
done.
checkout: 'HEAD'

etc... etc.. (neverending output ....)






## conda info & conda list

(bjorn3) bjorn@bjorn-ThinkPad-T450s:~/python_packages/pydna$ conda info
Current conda install:

               platform : linux-64
          conda version : 4.2.9
       conda is private : False
      conda-env version : 4.2.9
    conda-build version : 2.0.3
         python version : 2.7.12.final.0
       requests version : 2.11.1
       root environment : /home/bjorn/anaconda  (writable)
    default environment : /home/bjorn/anaconda/envs/bjorn3
       envs directories : /home/bjorn/anaconda/envs
          package cache : /home/bjorn/anaconda/pkgs
           channel URLs : https://repo.continuum.io/pkgs/free/linux-64/
                          https://repo.continuum.io/pkgs/free/noarch/
                          https://repo.continuum.io/pkgs/pro/linux-64/
                          https://repo.continuum.io/pkgs/pro/noarch/
                          https://conda.anaconda.org/anaconda/linux-64/
                          https://conda.anaconda.org/anaconda/noarch/
                          https://conda.anaconda.org/BjornFJohansson/linux-64/
                          https://conda.anaconda.org/BjornFJohansson/noarch/
            config file : /home/bjorn/.condarc
           offline mode : False

(bjorn3) bjorn@bjorn-ThinkPad-T450s:~/python_packages/pydna$ conda list
# packages in environment at /home/bjorn/anaconda/envs/bjorn3:
#
_license                  1.1                      py35_1    defaults
_nb_ext_conf              0.3.0                    py35_0    defaults
alabaster                 0.7.9                    py35_0    defaults
anaconda                  4.2.0               np111py35_0    defaults
anaconda-clean            1.0.0                    py35_0    defaults
anaconda-client           1.5.1                    py35_0    defaults
anaconda-navigator        1.3.1                    py35_0    defaults
appdirs                   1.4.0                     <pip>
argcomplete               1.0.0                    py35_1    defaults
astroid                   1.4.7                    py35_0    defaults
astropy                   1.2.1               np111py35_0    defaults
babel                     2.3.4                    py35_0    defaults
backports                 1.0                      py35_0    defaults
backports.shutil_get_terminal_size 1.0.0                    py35_0    conda-forge
beautifulsoup4            4.5.1                    py35_0    defaults
biopython                 1.67                np111py35_0    defaults
bitarray                  0.8.1                    py35_0    defaults
blaze                     0.10.1                   py35_0    defaults
bokeh                     0.12.2                   py35_0    defaults
boto                      2.42.0                   py35_0    defaults
bottleneck                1.1.0               np111py35_0    defaults
bumpversion               0.5.3                     <pip>
cairo                     1.12.18                       6    defaults
cffi                      1.7.0                    py35_0    defaults
chest                     0.2.3                    py35_0    defaults
click                     6.6                      py35_0    defaults
cloudpickle               0.2.1                    py35_0    defaults
clyent                    1.2.2                    py35_0    defaults
colorama                  0.3.7                    py35_0    defaults
configobj                 5.0.6                    py35_0    defaults
contextlib2               0.5.3                    py35_0    defaults
cryptography              1.5                      py35_0    defaults
curl                      7.49.0                        1    defaults
cycler                    0.10.0                   py35_0    defaults
cython                    0.24.1                   py35_0    defaults
cytoolz                   0.8.0                    py35_0    defaults
dask                      0.11.0                   py35_0    defaults
datashape                 0.5.2                    py35_0    defaults
dbus                      1.10.10                       0    defaults
decorator                 4.0.10                   py35_0    defaults
dill                      0.2.5                    py35_0    defaults
docopt                    0.6.2                    py35_0    conda-forge
docutils                  0.12                     py35_2    defaults
dropbox                   6.8.0                     <pip>
dynd-python               0.7.2                    py35_0    defaults
entrypoints               0.2.2                    py35_0    defaults
et_xmlfile                1.0.1                    py35_0    defaults
exec-wrappers             0.1.1                     <pip>
expat                     2.1.0                         0    defaults
fastcache                 1.0.2                    py35_1    defaults
filelock                  2.0.6                    py35_0    defaults
flask                     0.11.1                   py35_0    defaults
flask-cors                2.1.2                    py35_0    defaults
fontconfig                2.11.1                        6    defaults
freetype                  2.5.5                         1    defaults
get_terminal_size         1.0.0                    py35_0    defaults
gevent                    1.1.2                    py35_0    defaults
glib                      2.43.0                        1    defaults
gmp                       6.1.1                         0    conda-forge
greenlet                  0.4.10                   py35_0    defaults
gst-plugins-base          1.8.0                         0    defaults
gstreamer                 1.8.0                         0    defaults
h5py                      2.6.0               np111py35_2    defaults
harfbuzz                  0.9.39                        1    defaults
hdf5                      1.8.17                        1    defaults
heapdict                  1.0.0                    py35_1    defaults
icu                       54.1                          0    defaults
idna                      2.1                      py35_0    defaults
imagesize                 0.7.1                    py35_0    defaults
ipykernel                 4.5.0                     <pip>
ipykernel                 4.5.0                    py35_0    defaults
ipython                   5.1.0                    py35_0    defaults
ipython_genutils          0.1.0                    py35_0    defaults
ipywidgets                5.2.2                    py35_0    defaults
itsdangerous              0.24                     py35_0    defaults
jbig                      2.1                           0    defaults
jdcal                     1.2                      py35_1    defaults
jedi                      0.9.0                    py35_1    defaults
jinja2                    2.8                      py35_1    defaults
jpeg                      8d                            2    defaults
jsonschema                2.5.1                    py35_0    defaults
jupyter                   1.0.0                    py35_3    defaults
jupyter_client            4.4.0                    py35_0    defaults
jupyter_console           5.0.0                    py35_0    defaults
jupyter_core              4.2.0                    py35_0    defaults
jupyterlab                0.4.1                    py35_0    conda-forge
lazy-object-proxy         1.2.1                    py35_0    defaults
libdynd                   0.7.2                         0    defaults
libffi                    3.2.1                         0    defaults
libgcc                    4.8.5                         2    defaults
libgfortran               3.0.0                         1    defaults
libpng                    1.6.22                        0    defaults
libsodium                 1.0.10                        0    defaults
libtiff                   4.0.6                         2    defaults
libxcb                    1.12                          0    defaults
libxml2                   2.9.2                         0    defaults
libxslt                   1.1.28                        0    defaults
llvmlite                  0.13.0                   py35_0    defaults
locket                    0.2.0                    py35_1    defaults
lxml                      3.6.4                    py35_0    defaults
markupsafe                0.23                     py35_2    defaults
matplotlib                1.5.3               np111py35_0    defaults
mistune                   0.7.3                    py35_0    defaults
mkl                       11.3.3                        0    defaults
mkl-service               1.1.2                    py35_2    defaults
mpldatacursor             0.6.2                         0    BjornFJohansson
mpmath                    0.19                     py35_1    defaults
multipledispatch          0.4.8                    py35_0    defaults
nb_anacondacloud          1.2.0                    py35_0    defaults
nb_conda                  2.0.0                    py35_0    defaults
nb_conda_kernels          2.0.0                    py35_0    defaults
nbconvert                 4.2.0                    py35_0    defaults
nbformat                  4.1.0                    py35_0    defaults
nbformat                  4.1.0                     <pip>
nbpresent                 3.0.2                    py35_0    defaults
networkx                  1.11                     py35_0    defaults
nltk                      3.2.1                    py35_0    defaults
nose                      1.3.7                    py35_1    defaults
notebook                  4.2.3                    py35_0    defaults
notebook                  4.2.3                     <pip>
numba                     0.28.1              np111py35_0    defaults
numexpr                   2.6.1               np111py35_0    defaults
numpy                     1.11.1                   py35_0    defaults
odo                       0.5.0                    py35_1    defaults
openpyxl                  2.3.2                    py35_0    defaults
openssl                   1.0.2j                        0    defaults
ordered-set               2.0.1                    py35_0    conda-forge
pandas                    0.18.1              np111py35_0    defaults
pandoc                    1.17.2                        0    conda-forge
partd                     0.3.6                    py35_0    defaults
patchelf                  0.9                           0    defaults
path.py                   8.2.1                    py35_0    conda-forge
pathlib2                  2.1.0                    py35_0    defaults
patsy                     0.4.1                    py35_0    defaults
pep8                      1.7.0                    py35_0    defaults
percache                  0.3.0                     <pip>
pexpect                   4.0.1                    py35_0    defaults
pickleshare               0.7.4                    py35_0    defaults
pillow                    3.3.1                    py35_0    defaults
pint                      0.7.2                    py35_0    BjornFJohansson
pip                       8.1.2                    py35_0    defaults
pixman                    0.32.6                        0    defaults
pkginfo                   1.3.2                    py35_0    defaults
ply                       3.9                      py35_0    defaults
prettytable               0.7.2                     <pip>
prompt_toolkit            1.0.3                    py35_0    defaults
psutil                    4.3.1                    py35_0    defaults
ptyprocess                0.5.1                    py35_0    defaults
py                        1.4.31                   py35_0    defaults
pyasn1                    0.1.9                    py35_0    defaults
pycosat                   0.6.1                    py35_1    defaults
pycparser                 2.14                     py35_1    defaults
pycrypto                  2.6.1                    py35_4    defaults
pycurl                    7.43.0                   py35_0    defaults
pyflakes                  1.3.0                    py35_0    defaults
pygments                  2.1.3                    py35_0    defaults
pyinstaller               3.2.3                    py35_0    acellera
pylint                    1.5.4                    py35_1    defaults
pyopenssl                 16.0.0                   py35_0    defaults
pyparsing                 2.1.4                    py35_0    defaults
pyqt                      5.6.0                    py35_0    defaults
pytables                  3.2.3.1             np111py35_0    defaults
pytest                    2.9.2                    py35_0    defaults
python                    3.5.2                         0    defaults
python-dateutil           2.5.3                    py35_0    defaults
pytz                      2016.6.1                 py35_0    defaults
pyyaml                    3.12                     py35_0    defaults
pyzmq                     15.4.0                   py35_0    defaults
qt                        5.6.0                         0    defaults
qtawesome                 0.3.3                    py35_0    defaults
qtconsole                 4.2.1                    py35_1    defaults
qtpy                      1.1.2                    py35_0    defaults
readline                  6.2                           2    defaults
redis                     3.2.0                         0    defaults
redis-py                  2.10.5                   py35_0    defaults
reportlab                 3.3.0                    py35_0    defaults
requests                  2.11.1                   py35_0    defaults
rope                      0.9.4                    py35_1    defaults
ruamel_yaml               0.11.7                   py35_0    defaults
scikit-image              0.12.3              np111py35_1    defaults
scikit-learn              0.17.1              np111py35_2    defaults
scipy                     0.18.1              np111py35_0    defaults
setuptools                27.2.0                   py35_0    defaults
simplegeneric             0.8.1                    py35_1    defaults
singledispatch            3.4.0.3                  py35_0    defaults
sip                       4.18                     py35_0    defaults
six                       1.10.0                   py35_0    defaults
snowballstemmer           1.2.1                    py35_0    defaults
sockjs-tornado            1.0.3                    py35_0    defaults
sphinx                    1.4.6                    py35_0    defaults
sphinx_rtd_theme          0.1.9                    py35_0    defaults
spyder                    3.0.0                    py35_0    defaults
sqlalchemy                1.0.13                   py35_0    defaults
sqlite                    3.13.0                        0    defaults
statsmodels               0.6.1               np111py35_1    defaults
sympy                     1.0                      py35_0    defaults
terminado                 0.6                      py35_0    defaults
tk                        8.5.18                        0    defaults
toolz                     0.8.0                    py35_0    defaults
tornado                   4.4.2                     <pip>
tornado                   4.4.1                    py35_0    defaults
traitlets                 4.3.1                     <pip>
traitlets                 4.3.0                    py35_0    defaults
typing                    3.5.2.2                   <pip>
unicodecsv                0.14.1                   py35_0    defaults
urllib3                   1.16                     py35_1    conda-forge
versioneer                0.17                      <pip>
wcwidth                   0.1.7                    py35_0    defaults
werkzeug                  0.11.11                  py35_0    defaults
wheel                     0.29.0                   py35_0    defaults
widgetsnbextension        1.2.6                    py35_0    defaults
wrapt                     1.10.6                   py35_0    defaults
xlrd                      1.0.0                    py35_0    defaults
xlsxwriter                0.9.3                    py35_0    defaults
xlwt                      1.1.2                    py35_0    defaults
xz                        5.2.2                         0    conda-forge
yaml                      0.1.6                         0    defaults
zeromq                    4.1.4                         0    defaults
zlib                      1.2.8                         3    defaults
(bjorn3) bjorn@bjorn-ThinkPad-T450s:~/python_packages/pydna$ 



## [meta,yaml](https://github.com/BjornFJohansson/pydna/blob/py3/meta.yaml)

package:
    name: pydna
    version: {{ environ.get('GIT_DESCRIBE_TAG', 'default') }}

build:
  number: {{ environ.get('GIT_DESCRIBE_NUMBER', 0) }}

source:
  git_url: .

requirements:
    build:
        - python ==3.5
        - ipython
        - jupyter
        - biopython
        - networkx
        - appdirs
        - prettytable
        - numpy
        - scipy
        - matplotlib
        - mpldatacursor
        - pint
        - ordered-set
    run:
        - python ==3.5
        - ipython
        - jupyter
        - biopython
        - networkx
        - appdirs
        - prettytable
        - numpy
        - scipy
        - matplotlib
        - mpldatacursor
        - pint
        - ordered-set
about:
  home: https://github.com/BjornFJohansson/pydna
  license: BSD
  summary: Classes and code for representing double stranded DNA and functions for simulating homologous recombination and Gibson assembly.

```



Discussion:
Wow, that's an impressive pathology, and I'll try to reproduce it.  Otherwise, a couple of bits of advice:
- Never delete your pkgs directory.  Conda itself gets pretty confused when you do that.  It is especially important if any packages are symlinked rather than hardlinked.  This may be the root cause of your problem - some conda error that is getting swallowed, and rather than crashing out, it is retrying the build.

If you need to clean out the pkgs folder, use `conda clean -pt`
- You are pinning python very tightly there.  Why?  If you really need to skip other python builds, a better way is instead to have:

```
build:
  skip: True   #[py!=35]
```

This is because when you pin python, the python version does not always make it into the build string.  Known issue, needs resolution.

Also, if you're working more on building code rather than running it, I recommend using miniconda instead of anaconda.  It's just a lot smaller, and easier to delete/reinstall if/when you need to.

Anaconda is nice because it includes so many dependencies, but if you're not using them, then they are just taking time and space.

The clean install ended up working after all. I don't know what was wrong, but I also switched to Anaconda3 which has python3 in root. 

The question about pinning is a good one, I probably copied a meta.yaml from some other project when I made the transition from python 2 to 3. I removed it and it works fine and produces packages with the expected name.

Curiously, if I remove all run requirements from build requirements (as I understand it should only be python and setuptools), the build fails with some cryptic reference including paths with lots of _placeholder_ things in them.

Hi, I am having the same issue with a recent miniconda installation:

```
[jq2@merlot conda-recipes]$ conda list
# packages in environment at /lustre/pfs1/cades-bsd/jq2/miniconda-cades:
#
conda-build               2.0.0                    py27_0
conda                     4.1.11                   py27_1    conda-forge
conda-env                 2.5.2                    py27_0    conda-forge
filelock                  2.0.6                    py27_0    conda-forge
jinja2                    2.8                      py27_1    conda-forge
markupsafe                0.23                     py27_0    conda-forge
patchelf                  0.9                           1    conda-forge
pkginfo                   1.2.1                    py27_0    conda-forge
openssl                   1.0.2h                        1
pip                       8.1.2                    py27_0
pycosat                   0.6.1                    py27_1
pycrypto                  2.6.1                    py27_4
python                    2.7.12                        1
pyyaml                    3.11                     py27_4
readline                  6.2                           2
requests                  2.10.0                   py27_0
ruamel_yaml               0.11.14                  py27_0
setuptools                23.0.0                   py27_0
sqlite                    3.13.0                        0
tk                        8.5.18                        0
wheel                     0.29.0                   py27_0
yaml                      0.1.6                         0
zlib                      1.2.8                         3

[jq2@merlot conda-recipes]$ cat bridger/meta.yaml
about:
  home: 'http://BridgerRNASeq.sourceforge.net'
  license: GPLv2
  summary: 'Bridger is an efficient de novo transcriptome assembler for RNA-Seq data.'

build:
  number: 1
  skip: True # [osx]

requirements:
  build:
    - gcc
    - zlib
    - boost ==1.60
    - fastool
    - rsem
  run:
    - libgcc
    - zlib
    - boost ==1.60
    - fastool
    - rsem
package:
  name: bridger
  version: 'r2014.12.01'
source:
  #git_url: https://github.com/fmaguire/Bridger_Assembler
  url: http://tenet.dl.sourceforge.net/project/rnaseqassembly/Bridger_r2014-12-01.tar.gz
  fn: Bridger_r2014-12-01.tar.gz
  md5: 4f4a8a00363df197463259a7eef906b2
test:
  commands:
    - 'Assemble -h 2>&1 | grep "Usage: Assemble"'

[jq2@merlot conda-recipes]$ conda info
Current conda install:

             platform : linux-64
        conda version : 4.1.11
    conda-env version : 2.5.2
  conda-build version : 2.0.0
       python version : 2.7.12.final.0
     requests version : 2.10.0
     root environment : /lustre/pfs1/cades-bsd/jq2/miniconda-cades  (writable)
  default environment : /lustre/pfs1/cades-bsd/jq2/miniconda-cades
     envs directories : /lustre/pfs1/cades-bsd/jq2/miniconda-cades/envs
        package cache : /lustre/pfs1/cades-bsd/jq2/miniconda-cades/pkgs
         channel URLs : https://conda.anaconda.org/conda-forge/linux-64/
                        https://conda.anaconda.org/conda-forge/noarch/
                        https://conda.anaconda.org/bioconda/linux-64/
                        https://conda.anaconda.org/bioconda/noarch/
                        https://repo.continuum.io/pkgs/free/linux-64/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/linux-64/
                        https://repo.continuum.io/pkgs/pro/noarch/
          config file : /lustre/pfs1/cades-bsd/jq2/.condarc
         offline mode : False
    is foreign system : False

```

I started with conda-build version 2.0.4, then went through all the version up to 2.0.0. Though, I don't think changing the version did anything. According to conda list I have 2.0.0 installed, but if I look at:

```
[jq2@merlot conda-recipes]$ conda-build --version
conda-build 2.0.4
```

I see the version has not changed...weird. But the problem overall is that the index loops forever:

```
[jq2@merlot conda-recipes]$ conda build bridger
Removing old build environment
BUILD START: bridger-r2014.12.01-1
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/linux-64
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/noarch
# If you want to upload this package to anaconda.org later, type:
#
# $ anaconda upload /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/linux-64/bridger-r2014.12.01-1.tar.bz2
#
# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

Removing old build environment
BUILD START: bridger-r2014.12.01-1
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/linux-64
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/noarch
# If you want to upload this package to anaconda.org later, type:
#
# $ anaconda upload /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/linux-64/bridger-r2014.12.01-1.tar.bz2
#
# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

Removing old build environment
BUILD START: bridger-r2014.12.01-1
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/linux-64
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/noarch
# If you want to upload this package to anaconda.org later, type:
#
# $ anaconda upload /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/linux-64/bridger-r2014.12.01-1.tar.bz2
#
# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

Removing old build environment
BUILD START: bridger-r2014.12.01-1
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/linux-64
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/noarch
# If you want to upload this package to anaconda.org later, type:
#
# $ anaconda upload /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/linux-64/bridger-r2014.12.01-1.tar.bz2
#
# To have conda build upload to anaconda.org automatically, use
# $ conda config --set anaconda_upload yes

Removing old build environment
BUILD START: bridger-r2014.12.01-1
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/linux-64
updating index in: /lustre/pfs1/cades-bsd/jq2/miniconda-cades/conda-bld/noarch
```

Any advice?

P

@PietJones conda-build is failing to die when it can't obtain or build some dependency.  This may be due to circular dependencies in your build requirements.  To diagnose the issue, please try to install each of your build requirements one-by-one.  If any are not built, you can try to build them.  You may discover a circular dependency this way.  If all of them succeed at building, then I'm pretty sure your top level package will also then succeed.

Hi,

Thanks for the advice, I stripped down the dependencies as you suggested
and ran build on them each one individually. I believe I found the problem,
building the packages rsem, had r as a dependency, but the r channel was
not added. Which made resulted in the infinite cycle, not sure if that is
an intended side-effect. Anyway, solved this by:

conda config --add channels r

The building is continuing, now I have to solve other problems :)

Thanks,
P

On Tue, Oct 11, 2016 at 1:15 PM, Mike Sarahan notifications@github.com
wrote:

> @PietJones https://github.com/PietJones conda-build is failing to die
> when it can't obtain or build some dependency. This may be due to circular
> dependencies in your build requirements. To diagnose the issue, please try
> to install each of your build requirements one-by-one. If any are not
> built, you can try to build them. You may discover a circular dependency
> this way. If all of them succeed at building, then I'm pretty sure your top
> level package will also then succeed.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> https://github.com/conda/conda-build/issues/1432#issuecomment-252982473,
> or mute the thread
> https://github.com/notifications/unsubscribe-auth/ASTyvnw3_RFIzFHGN6fsJx4IX6vr09EWks5qy8QkgaJpZM4KOpwO
> .

@BjornFJohansson, I tried to reproduce using your recipe, but get:

```
BUILD START: pydna-default-py27_0
    (actual version deferred until further download or env creation)
updating index in: /opt/miniconda/conda-bld/linux-64
updating index in: /opt/miniconda/conda-bld/noarch
Can't build /home/dev/code/rtest due to unsatisfiable dependencies:
Packages missing in current linux-64 channels:
  - prettytable
  - ordered-set
```

As I would hope.  I've tried on both mac and Linux (docker).

@PietJones, same story.  With your recipe, I get:

```
BUILD START: bridger-r2014.12.01-1
updating index in: /opt/miniconda/conda-bld/linux-64
updating index in: /opt/miniconda/conda-bld/noarch
Can't build /home/dev/code/rtest due to unsatisfiable dependencies:
Packages missing in current linux-64 channels:
  - fastool
  - rsem
```

Do either of you have those recipes in the same folder with your recipe that you're trying to build?  I do not, but that might be a different code path.

I have those packages in my channel [BjornFJohansson](https://anaconda.org/bjornfjohansson).
I dont have the problem anymore, I did a switch to anaconda with python 3 in the root.

Hi,

My solution was to do:

conda config --add channels bioconda #fastool and rsem are recipes in this
channel
conda config --add channels r   #rsem needs this, as it has a dependency
namely r (this was the one that causes the infinite loop).
conda build bridger

Though the testing of the package works, the program itself won't run.
Waiting for someone to reply to the message I left asking about conda-build
structure and best suggestions. (The program has weird subfolders that are
needed by the perl script that launches it, not sure how to include those
subfolders when building and installing).

So to reproduce, maybe try and add only bioconda, and not r as a channel.

P

On Thu, Oct 13, 2016 at 11:04 AM, Björn Johansson notifications@github.com
wrote:

> I have those packages in my channel BjornFJohansson
> https://anaconda.org/bjornfjohansson.
> I dont have the problem anymore, I did a switch to anaconda with python 3
> in the root.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> https://github.com/conda/conda-build/issues/1432#issuecomment-253540485,
> or mute the thread
> https://github.com/notifications/unsubscribe-auth/ASTyvt05tpMwddKDHjCMHUGL9E6W65_mks5qzkhygaJpZM4KOpwO
> .

Thank you Piet, I was able to reproduce the behavior by adding the bioconda channel.

For your question about the program not running - I don't know what message you're referring to.  Was it on the conda mailing list?  Was it an issue here?  Please link.

In general, if you have subfolders that are needed, you need to copy or create them in build.sh.  There is one catch: conda-build will not keep empty folders during packaging.  You may need to place dummy files in created folders to ensure that they exist at install time.

Hi,

Pleasure, glad I can help uncovering any possible bugs or shed light on
things.

Here is the link to the post I talked about:
https://groups.google.com/a/continuum.io/forum/#!topic/conda/j7dIZ1cOoFY

I will try creating the folders thanks, where will these end up when I
actually run install? As for some reason the perl script has the location
hard coded as a variable (well, relative to the perl script actually...no
not hardcoded...)

P

On Thu, Oct 13, 2016 at 11:24 AM, Mike Sarahan notifications@github.com
wrote:

> Thank you Piet, I was able to reproduce the behavior by adding the
> bioconda channel.
> 
> For your question about the program not running - I don't know what
> message you're referring to. Was it on the conda mailing list? Was it an
> issue here? Please link.
> 
> In general, if you have subfolders that are needed, you need to copy or
> create them in build.sh. There is one catch: conda-build will not keep
> empty folders during packaging. You may need to place dummy files in
> created folders to ensure that they exist at install time.
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> https://github.com/conda/conda-build/issues/1432#issuecomment-253546690,
> or mute the thread
> https://github.com/notifications/unsubscribe-auth/ASTyvpawED16kOOegxabzp8bDz-zDeGYks5qzk0xgaJpZM4KOpwO
> .


Resolving patch:
diff --git a/conda_build/build.py b/conda_build/build.py
--- a/conda_build/build.py
+++ b/conda_build/build.py
@@ -36,7 +36,8 @@
 from .conda_interface import package_cache
 from .conda_interface import prefix_placeholder, linked, symlink_conda
 from .conda_interface import url_path
-from .conda_interface import Resolve, MatchSpec, NoPackagesFound, Unsatisfiable
+from .conda_interface import (Resolve, MatchSpec, NoPackagesFound, Unsatisfiable,
+                              CondaValueError, NoPackagesFoundError)
 from .conda_interface import TemporaryDirectory
 from .conda_interface import get_rc_urls, get_local_urls
 from .conda_interface import VersionOrder
@@ -559,7 +560,7 @@ def get_conda_build_index_versions(index):
     pkgs = []
     try:
         pkgs = r.get_pkgs(MatchSpec('conda-build'))
-    except NoPackagesFound:
+    except (NoPackagesFound, NoPackagesFoundError):
         log.warn("Could not find any versions of conda-build in the channels")
     return [pkg.version for pkg in pkgs]
 
@@ -739,8 +740,9 @@ def build(m, config, post=None, need_source_download=True, need_reparse_in_env=F
                                 data = open(work_file).read()
                             with open(work_file, 'w') as bf:
                                 bf.write('source "{conda_root}activate" "{build_prefix}" &> '
-                                         '/dev/null\n'.format(conda_root=root_script_dir + os.path.sep,
-                                                         build_prefix=config.build_prefix))
+                                         '/dev/null\n'.format(conda_root=root_script_dir +
+                                                              os.path.sep,
+                                                              build_prefix=config.build_prefix))
                                 bf.write(data)
                         else:
                             if not isfile(work_file):
@@ -1050,6 +1052,7 @@ def build_tree(recipe_list, config, build_only=False, post=False, notest=False,
         # delete_trash(None)
 
     already_built = set()
+    extra_help = ""
     while recipe_list:
         # This loop recursively builds dependencies if recipes exist
         if build_only:
@@ -1092,13 +1095,8 @@ def build_tree(recipe_list, config, build_only=False, post=False, notest=False,
                                    config=recipe_config)
                 if not notest and ok_to_test:
                     test(metadata, config=recipe_config)
-        except (NoPackagesFound, Unsatisfiable) as e:
+        except (NoPackagesFound, NoPackagesFoundError, Unsatisfiable, CondaValueError) as e:
             error_str = str(e)
-            # Typically if a conflict is with one of these
-            # packages, the other package needs to be rebuilt
-            # (e.g., a conflict with 'python 3.5*' and 'x' means
-            # 'x' isn't build for Python 3.5 and needs to be
-            # rebuilt).
             skip_names = ['python', 'r']
             add_recipes = []
             # add the failed one back in at the beginning - but its deps may come before it
@@ -1108,12 +1106,17 @@ def build_tree(recipe_list, config, build_only=False, post=False, notest=False,
                     continue
                 pkg = line.lstrip('  - ').split(' -> ')[-1]
                 pkg = pkg.strip().split(' ')[0]
-                if pkg in skip_names:
-                    continue
 
                 if pkg in to_build_recursive:
                     raise RuntimeError("Can't build {0} due to unsatisfiable dependencies:\n"
-                                       .format(recipe) + error_str)
+                                       .format(recipe) + error_str + "\n" + extra_help)
+
+                if pkg in skip_names:
+                    to_build_recursive.append(pkg)
+                    extra_help = """Typically if a conflict is with the Python or R
+packages, the other package needs to be rebuilt
+(e.g., a conflict with 'python 3.5*' and 'x' means
+'x' isn't build for Python 3.5 and needs to be rebuilt."""
 
                 recipe_glob = glob(os.path.join(recipe_parent_dir, pkg))
                 if recipe_glob:
@@ -1125,7 +1128,7 @@ def build_tree(recipe_list, config, build_only=False, post=False, notest=False,
                         add_recipes.append(recipe_dir)
                 else:
                     raise RuntimeError("Can't build {0} due to unsatisfiable dependencies:\n"
-                                       .format(recipe) + error_str)
+                                       .format(recipe) + error_str + "\n\n" + extra_help)
             recipe_list.extendleft(add_recipes)
 
         # outputs message, or does upload, depending on value of args.anaconda_upload
diff --git a/conda_build/conda_interface.py b/conda_build/conda_interface.py
--- a/conda_build/conda_interface.py
+++ b/conda_build/conda_interface.py
@@ -18,7 +18,7 @@
                            rm_rf, symlink_conda, rm_fetched, package_cache)  # NOQA
 from conda.lock import Locked  # NOQA
 from conda.misc import untracked, walk_prefix  # NOQA
-from conda.resolve import MatchSpec, NoPackagesFound, Resolve, Unsatisfiable, normalized_version  # NOQA
+from conda.resolve import MatchSpec, NoPackagesFound, Resolve, Unsatisfiable, normalized_version, CondaValueError  # NOQA
 from conda.signature import KEYS, KEYS_DIR, hash_file, verify  # NOQA
 from conda.utils import human_bytes, hashsum_file, md5_file, memoized, unix_path_to_win, win_path_to_unix, url_path  # NOQA
 import conda.config as cc  # NOQA
@@ -52,6 +52,7 @@
     load_condarc = lambda fn: conda.base.context.reset_context([fn])
     PaddingError = conda.exceptions.PaddingError
     LinkError = conda.exceptions.LinkError
+    NoPackagesFoundError = conda.exceptions.NoPackagesFoundError
 
 else:
     from conda.config import get_default_urls, non_x86_linux_machines, load_condarc  # NOQA
@@ -78,6 +79,9 @@ class PaddingError(Exception):
     class LinkError(Exception):
         pass
 
+    class NoPackagesFoundError(Exception):
+        pass
+
 
 class SignatureError(Exception):
     pass
diff --git a/conda_build/utils.py b/conda_build/utils.py
--- a/conda_build/utils.py
+++ b/conda_build/utils.py
@@ -147,7 +147,7 @@ def copytree(src, dst, symlinks=False, ignore=None, dry_run=False):
                     mode = stat.S_IMODE(st.st_mode)
                     os.lchmod(d, mode)
                 except:
-                    pass # lchmod not available
+                    pass  # lchmod not available
             elif os.path.isdir(s):
                 copytree(s, d, symlinks, ignore)
             else:


Test patch: