Repository: numpy/numpy

Issue:
einsum 'optimal' optimization worse than 'greedy' optimization in at least one case
I find the following example where einsum_path is able to find a better contraction ordering with the optimize keyword set to 'greedy' instead of 'optimal'. I believe that optimal uses a brute force enumeration of all possibilities, so this in principle shouldn't happen. 

I'm using numpy 1.13.1 running on macOS 10.12.6 with python 3.5.2. Attached you can see the behavior I'm describing.

The einsum path using greedy optimization:
![image](https://user-images.githubusercontent.com/19966115/30818664-30aad020-a1ea-11e7-84e5-c9cb99fed1bb.png)

The einsum path using optimal optimization:
![image](https://user-images.githubusercontent.com/19966115/30818718-5c03edba-a1ea-11e7-8b3e-31fb209d436b.png)


Discussion:
Can paste that as text, so can we can run it?
```python
y = np.random.rand(100)
X = np.random.rand(100,100)
print(np.einsum_path('a,b,c,d,e,ab,ac,ad,bc,bd,cd,ce', *[y]*5, *[X]*7, optimize='optimal')[1])

y = np.random.rand(100)
X = np.random.rand(100,100)
print(np.einsum_path('a,b,c,d,e,ab,ac,ad,bc,bd,cd,ce', *[y]*5, *[X]*7, optimize='greedy')[1])
```


@dgasmith Thoughts?
Whoops, I see the error. I didn't handle the case where the `optimal` algorithm would need to do multiple terms at once *without* reaching the end of the run. A small fix and we now obtain the following for the `optimal` algorithm:

```
  Complete contraction:  a,b,c,d,e,ab,ac,ad,bc,bd,cd,ce->
         Naive scaling:  5
     Optimized scaling:  4
      Naive FLOP count:  2.200e+11
  Optimized FLOP count:  2.001e+08
   Theoretical speedup:  1099.670
  Largest intermediate:  1.000e+04 elements
--------------------------------------------------------------------------
scaling                  current                                remaining
--------------------------------------------------------------------------
   2                     b,a->ab          c,d,e,ab,ac,ad,bc,bd,cd,ce,ab->
   2                    ad,d->ad            c,e,ab,ac,bc,bd,cd,ce,ab,ad->
   2                     ce,e->c               c,ab,ac,bc,bd,cd,ab,ad,c->
   1                      c,c->c                 ab,ac,bc,bd,cd,ab,ad,c->
   2                   ab,ab->ab                    ac,bc,bd,cd,ad,c,ab->
   2                    c,ac->ac                      bc,bd,cd,ad,ab,ac->
   4         ac,ab,ad,cd,bd,bc->                                       ->
```

This path is equivalent to greedy.

As a note the `optimal` path scales factorially and quickly can take impractical amounts of time. We could add more filtering to speed up the computation, but that won't push the factorial scaling wall too far back.

We should probably add a stop that if a path iteration is going to take more than say a minute (~20M path trials) the `optimal` path stops trying. Is there a good way to let the user know that optimal is giving up and perhaps defaulting back to a `greedy` algorithm?

@dgasmith - I think it might just be enough to document clearly that `optimal` can take very long.
@mhvk Ah, perhaps I should give a example. Napkin math shows the OP's example would take ~16 years if there was no truncation in iterations (as shown here) or sieving based on size.
@dgasmith - I guess my main concern was about imposing an arbitrary "this is too complex" limit, and thus adding complexity. But obviously 16 years is a bit long by any standard... Anyway, perhaps most important to get your fix in!

Resolving patch:
diff --git a/numpy/core/einsumfunc.py b/numpy/core/einsumfunc.py
--- a/numpy/core/einsumfunc.py
+++ b/numpy/core/einsumfunc.py
@@ -166,8 +166,14 @@ def _optimal_path(input_sets, output_set, idx_dict, memory_limit):
                 new_pos = positions + [con]
                 iter_results.append((new_cost, new_pos, new_input_sets))
 
-        # Update list to iterate over
-        full_results = iter_results
+        # Update combinatorial list, if we did not find anything return best
+        # path + remaining contractions
+        if iter_results:
+            full_results = iter_results
+        else:
+            path = min(full_results, key=lambda x: x[0])[1]
+            path += [tuple(range(len(input_sets) - iteration))]
+            return path
 
     # If we have not found anything return single einsum contraction
     if len(full_results) == 0:


Test patch:
diff --git a/numpy/core/tests/test_einsum.py b/numpy/core/tests/test_einsum.py
--- a/numpy/core/tests/test_einsum.py
+++ b/numpy/core/tests/test_einsum.py
@@ -767,13 +767,13 @@ def test_random_cases(self):
 
 
 class TestEinSumPath(object):
-    def build_operands(self, string):
+    def build_operands(self, string, size_dict=global_size_dict):
 
         # Builds views based off initial operands
         operands = [string]
         terms = string.split('->')[0].split(',')
         for term in terms:
-            dims = [global_size_dict[x] for x in term]
+            dims = [size_dict[x] for x in term]
             operands.append(np.random.rand(*dims))
 
         return operands
@@ -863,6 +863,16 @@ def test_edge_paths(self):
         path, path_str = np.einsum_path(*edge_test4, optimize='optimal')
         self.assert_path_equal(path, ['einsum_path', (1, 2), (0, 2), (0, 1)])
 
+        # Edge test5
+        edge_test4 = self.build_operands('a,ac,ab,ad,cd,bd,bc->',
+                                         size_dict={"a": 20, "b": 20, "c": 20, "d": 20})
+        path, path_str = np.einsum_path(*edge_test4, optimize='greedy')
+        self.assert_path_equal(path, ['einsum_path', (0, 1), (0, 1, 2, 3, 4, 5)])
+
+        path, path_str = np.einsum_path(*edge_test4, optimize='optimal')
+        self.assert_path_equal(path, ['einsum_path', (0, 1), (0, 1, 2, 3, 4, 5)])
+
+
     def test_path_type_input(self):
         # Test explicit path handeling
         path_test = self.build_operands('dcc,fce,ea,dbf->ab')