Repository: conda/conda

Issue:
conda 4.0.2 failure during dependency resolution
This command fails in Linux and OS X

```
conda create -n testconda anaconda accelerate tornado=4.0.2
```

with this traceback:

```
Traceback (most recent call last):
  File "/Users/ewelch/miniconda/bin/conda", line 6, in <module>
    sys.exit(main())
  File "/Users/ewelch/miniconda/lib/python2.7/site-packages/conda/cli/main.py", line 139, in main
    args_func(args, p)
  File "/Users/ewelch/miniconda/lib/python2.7/site-packages/conda/cli/main.py", line 146, in args_func
    args.func(args, p)
  File "/Users/ewelch/miniconda/lib/python2.7/site-packages/conda/cli/main_create.py", line 49, in execute
    install.install(args, parser, 'create')
  File "/Users/ewelch/miniconda/lib/python2.7/site-packages/conda/cli/install.py", line 334, in install
    update_deps=args.update_deps)
  File "/Users/ewelch/miniconda/lib/python2.7/site-packages/conda/plan.py", line 435, in install_actions
    smh = r.dependency_sort(must_have)
  File "/Users/ewelch/miniconda/lib/python2.7/site-packages/conda/resolve.py", line 730, in dependency_sort
    depends = lookup(value)
  File "/Users/ewelch/miniconda/lib/python2.7/site-packages/conda/resolve.py", line 725, in lookup
    return set(ms.name for ms in self.ms_depends(value + '.tar.bz2'))
  File "/Users/ewelch/miniconda/lib/python2.7/site-packages/conda/resolve.py", line 574, in ms_depends
    deps = [MatchSpec(d) for d in self.index[fn].get('depends', [])]
KeyError: u'anaconda-2.1.0-np19py27_0.tar..tar.bz2'
```

Here is my `conda info`:

```
             platform : osx-64
        conda version : 4.0.2
  conda-build version : 1.18.1
       python version : 2.7.11.final.0
     requests version : 2.9.1
     root environment : /Users/ewelch/miniconda  (writable)
  default environment : /Users/ewelch/miniconda
     envs directories : /Users/ewelch/miniconda/envs
        package cache : /Users/ewelch/miniconda/pkgs
         channel URLs : https://conda.binstar.org/javascript/osx-64/
                        https://conda.binstar.org/javascript/noarch/
                        https://conda.binstar.org/wakari/osx-64/
                        https://conda.binstar.org/wakari/noarch/
                        https://repo.continuum.io/pkgs/free/osx-64/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/osx-64/
                        https://repo.continuum.io/pkgs/pro/noarch/
          config file : /Users/ewelch/.condarc
    is foreign system : False
```

I can circumvent the exception by using the following diff.  I don't propose that this is the solution (I don't know the code well enough), but I hope it's a useful starting point:

``` diff
diff --git a/conda/plan.py b/conda/plan.py
index a7fd836..519dedf 100644
--- a/conda/plan.py
+++ b/conda/plan.py
@@ -412,7 +412,7 @@ def install_actions(prefix, index, specs, force=False, only_names=None,

     pkgs = r.install(specs, [d + '.tar.bz2' for d in linked], update_deps=update_deps)
     for fn in pkgs:
-        dist = fn[:-8]
+        dist = fn[:fn.rindex('.tar.bz2')]
         name = install.name_dist(dist)
         if not name or only_names and name not in only_names:
             continue
```

The reason this fix works is that the filename, `fn`, is `anaconda-2.1.0-np19py27_0.tar.bz2[mkl]`, and the code assumes it ends in `.tar.bz2`.



Discussion:
@mcg1969  First time I'm seeing this.  Guessing this problem would still exist in the `4.0.x` branch?

@eriknw Thanks for the awesome report and digging in with a workable patch.

Yes, the problem still exists in the `4.0.x` branch.

Thanks Eric! I will look at this in the morning

A quick bifurcation reveals this commit is the culprit: 7d1f930

I'd say that just revealed the bug, it didn't cause it.

Odd. I can't reproduce it in my test notebook. Erik, can you do a `--debug` run and post the output here?

Nevermind, I have it.

Did I tell anyone I hate `with_features_depends` with a white hot burning hate? I do you know.

Great, good luck.

If we're looking for a commit to blame, this one is highly suspect to me:
https://github.com/conda/conda/commit/b5232f08ba4c3f5041aed70d54af3d25607102e8#diff-52a8935af92ea699b0cbb5feb84c19a2R445

The `for` loop linked above assigns names that include features, which will create names like `anaconda-2.1.0-np19py27_0.tar.bz2[mkl]`.  This breaks the assumption (revealed in the diff in the OP) that the key ends with `".tar.bz2"`.

After we make it through 4.0, I'd like to consider either clarification or alternatives to how we currently use features.  I think @mcg1969 would +1.

Oh, and we need a data model and real package objects.  This string parsing thing is beyond ridiculous.  `4.?`

I'm 100% with you. 

Here's the story. `with_features_depends` is a pain in the butt to deal with. Basically, it takes two (or more) different packages---one with features, one without---and combines them together into one metadata blob. To cleanly handle both, I create two separate packages in the index. The `[mkl]` suffix is the featured version.

For some reason I've lost the code that ensures these feature suffixes don't make it out into the wild.

Of course, with a proper data model, where we trade Package object pointers instead of filenames pretending to be pointers, we won't have this problem. These two "virtual" packages will have the same _internal_ filename, but will be unique in the eyes of the solver.


Resolving patch:
diff --git a/conda/resolve.py b/conda/resolve.py
--- a/conda/resolve.py
+++ b/conda/resolve.py
@@ -1014,8 +1014,13 @@ def clean(sol):
                      dashlist(', '.join(diff) for diff in diffs),
                      '\n  ... and others' if nsol > 10 else ''))
 
+            def stripfeat(sol):
+                return sol.split('[')[0]
             stdoutlog.info('\n')
-            return list(map(sorted, psolutions)) if returnall else sorted(psolutions[0])
+            if returnall:
+                return [sorted(map(stripfeat, psol)) for psol in psolutions]
+            else:
+                return sorted(map(stripfeat, psolutions[0]))
         except:
             stdoutlog.info('\n')
             raise


Test patch: