Repository: conda/conda

Issue:
update --all broken in 4.4.0rc
In 4.4.0rc an `update --all` wants to *remove* 90% of my packages. IMO an update should never remove any packages so the current behaviour seems pretty broken to me.

```
λ conda update --all
Solving environment: done

## Package Plan ##

  environment location: C:\Miniconda3


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    openssl-1.0.2m             |           vc14_0         5.4 MB  conda-forge

The following packages will be REMOVED:

    _license:            1.1-py36_1
    altair:              1.2.1-py_0                             conda-forge
    anaconda-client:     1.6.5-py36hd36550c_0                   https://repo.continuum.io/pkgs/main
    anaconda-navigator:  1.6.9-py36hc720852_0                   https://repo.continuum.io/pkgs/main
    anaconda-project:    0.8.2-py36_0                           conda-forge
    astroid:             1.5.3-py36h9d85297_0                   https://repo.continuum.io/pkgs/main
    astropy:             2.0.2-py36h06391c4_4                   https://repo.continuum.io/pkgs/main
<snip>
```


Discussion:
Can you give me the output of `conda list` for your environment so I can reproduce?
The result of `conda env export` is available at:
https://gist.github.com/dhirschfeld/0d8f57906980e90468f9060932d7962a

NB: internal channels have been removed since they're not reproducible but they may potentially play a part in the update plan? The internal channels are just a semi-up-to-date mirror of the continuum repos. In my dev env our internal code is all symlinked from git checkouts rather than installed as conda packages.
To help debugging I've posted the `-vvv` output at:
https://gist.github.com/dhirschfeld/43bd8019f1450e3a04f88b68275f734a

Can you repro this behaviour? I note the fix is scheduled for 4.4.1. At least for me, this bug means I'll have to skip 4.4.0 as `update --all` is a fundamental workflow for me to keep all my packages up to date. 
Probably worth a different issue, but `--force` doesn't seem to do what it used to pre-4.4
```
λ conda install conda=4.3* --force
Solving environment: done

## Package Plan ##

  environment location: C:\Miniconda3

  added / updated specs:
    - conda=4.3


The following packages will be UPDATED:

    gflags:    2.2.0-vc14_0                            [vc14] --> 2.2.0-vc14_1          conda-forge [vc14]
    hdf5:      1.10.1-h98b8871_1                       --> 1.10.1-vc14_1         conda-forge [vc14]
    libtiff:   4.0.9-h0f13578_0                        --> 4.0.9-vc14_0          conda-forge [vc14]

The following packages will be DOWNGRADED:

    boost:     1.65.1-py36_vc14h50d208f_3              [vc14] --> 1.65.1-py36_vc14_0    conda-forge [vc14]
    boost-cpp: 1.65.1-vc14h7942e50_3                   [vc14] --> 1.65.1-vc14_1         conda-forge [vc14]
    conda:     4.4.0rc2-py36h5d836c4_0    conda-canary --> 4.3.30-py36h7e176b0_0
    curl:      7.55.1-h3b839b5_4                       --> 7.55.1-vc14_0         conda-forge [vc14]
    hdf4:      4.2.13-h712560f_2                       --> 4.2.13-vc14_0         conda-forge [vc14]
    icu:       58.2-ha66f8fd_1                         --> 58.2-vc14_0           conda-forge [vc14]
    krb5:      1.14.2-h63dfc2a_6                       --> 1.14.2-vc14_0         conda-forge [vc14]
    libnetcdf: 4.5.0-hd92ae6e_5                        --> 4.5.0-vc14_3          conda-forge [vc14]
    libssh2:   1.8.0-h75d9407_3                        --> 1.8.0-vc14_2          conda-forge [vc14]
    snappy:    1.1.7-h777316e_3                        --> 1.1.7-vc14_1          conda-forge [vc14]
    yaml:      0.1.7-hc54c509_2                        --> 0.1.7-vc14_0          conda-forge [vc14]

Proceed ([y]/n)? n
```

i.e. I *just* wanted to downgrade conda without touching any other packages - previously exactly what `--force` was useful for. Now, conda wants to muck with my environment despite the `--force` flag being present.

Is this a new bug in 4.4?
Also, the final downgrade threw an error:
```
The following packages will be DOWNGRADED:

    conda: 4.4.0rc2-py36h5d836c4_0 conda-canary --> 4.3.30-py36h7e176b0_0

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
The batch file cannot be found.
C:\Miniconda3\Library\bin\conda.bat
At line:1 char:1
+ conda install conda=4.3* --force --override-channels -c defaults
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (:) [], FileNotFoundException
    + FullyQualifiedErrorId : System.IO.FileNotFoundException
```
...but the downgrade did seem to work fine.
#6283 seems to be the same underlying issue with 4.4.

Resolving patch:
diff --git a/conda/core/solve.py b/conda/core/solve.py
--- a/conda/core/solve.py
+++ b/conda/core/solve.py
@@ -143,20 +143,34 @@ def solve_final_state(self, deps_modifier=NULL, prune=NULL, ignore_pinned=NULL,
         # `solution` and `specs_map` are mutated throughout this method
         prefix_data = PrefixData(self.prefix)
         solution = tuple(Dist(d) for d in prefix_data.iter_records())
+        specs_from_history_map = History(self.prefix).get_requested_specs_map()
         if prune or deps_modifier == DepsModifier.UPDATE_ALL:
-            # start with empty specs map for UPDATE_ALL because we're optimizing the update
-            # only for specs the user has requested; it's ok to remove dependencies
+            # Start with empty specs map for UPDATE_ALL because we're optimizing the update
+            # only for specs the user has requested; it's ok to remove dependencies.
             specs_map = odict()
+
+            # However, because of https://github.com/conda/constructor/issues/138, we need
+            # to hard-code keeping conda, conda-build, and anaconda, if they're already in
+            # the environment.
+            solution_pkg_names = set(d.name for d in solution)
+            ensure_these = (pkg_name for pkg_name in {
+                'anaconda', 'conda', 'conda-build',
+            } if pkg_name not in specs_from_history_map and pkg_name in solution_pkg_names)
+            for pkg_name in ensure_these:
+                specs_from_history_map[pkg_name] = MatchSpec(pkg_name)
         else:
             specs_map = odict((d.name, MatchSpec(d.name)) for d in solution)
 
         # add in historically-requested specs
-        specs_from_history_map = History(self.prefix).get_requested_specs_map()
         specs_map.update(specs_from_history_map)
 
         # let's pretend for now that this is the right place to build the index
-        prepared_specs = tuple(concatv(specs_to_remove, specs_to_add,
-                                       itervalues(specs_from_history_map)))
+        prepared_specs = set(concatv(
+            specs_to_remove,
+            specs_to_add,
+            itervalues(specs_from_history_map),
+        ))
+
         index, r = self._prepare(prepared_specs)
 
         if specs_to_remove:
@@ -247,6 +261,12 @@ def solve_final_state(self, deps_modifier=NULL, prune=NULL, ignore_pinned=NULL,
         if deps_modifier == DepsModifier.UPDATE_ALL:
             specs_map = {pkg_name: MatchSpec(spec.name, optional=spec.optional)
                          for pkg_name, spec in iteritems(specs_map)}
+            # The anaconda spec is a special case here, because of the 'custom' version.
+            # Because of https://github.com/conda/conda/issues/6350, and until we implement
+            # something like https://github.com/ContinuumIO/anaconda-issues/issues/4298, I think
+            # this is the best we're going to do.
+            if 'anaconda' in specs_map:
+                specs_map['anaconda'] = MatchSpec('anaconda>1')
 
         # As a business rule, we never want to update python beyond the current minor version,
         # unless that's requested explicitly by the user (which we actively discourage).


Test patch: