Repository: conan-io/conan

Issue:
Remote regression from 1.8.4 to 1.9.0, multiple remotes hit downloading a package
To help us debug your issue please explain:

- [* ] I've read the [CONTRIBUTING guide](https://raw.githubusercontent.com/conan-io/conan/develop/.github/CONTRIBUTING.md).
- [* ] I've specified the Conan version, operating system version and any tool that can be relevant.
- [*] I've explained the steps to reproduce the error or the motivation/use case of the question/suggestion.

As before, I have a particular centos5 slave that doesn't have an upgraded ssl and really doesn't like it when conan tries to talk to bintray (crash). Machine is configured with an artifactory remote first, bintray second, but bintray remote should never be hit as all packages are in artifactory.

$ conan remote list
conan-local: https://artifactory.<removed>/api/conan/conan-local [Verify SSL: True]
conan-center: https://conan.bintray.com [Verify SSL: True]

We then `$ conan install . --build=missing` 

Now in below example libmagic is the first full hit where binaries are available for this system ("Download"). gtest is package only (being built). 

i.e.
```
13:23:49 [linux_i686]     bzip2/1.0.6@jenkins/stable:1951ee17b0cc43d5963ddb6e1174db19bcc85137 - Build
13:23:49 [linux_i686]     gtest/1.8.0@jenkins/stable:caef3bc369421e559b530d159bebf04d0448c076 - Build
13:23:49 [linux_i686]     libmagic/5.33@jenkins/stable:2add8c254ae251eda8b741b469172d309bd42b7d - Download
```
Anyway, snip of the logs
```
[linux_x86_64] gtest/1.8.0@jenkins/stable: Not found in local cache, looking in remotes...
[linux_x86_64] gtest/1.8.0@jenkins/stable: Trying with 'conan-local'...
[linux_x86_64] Downloading conanmanifest.txt
[linux_x86_64]
[linux_x86_64] Downloading conanfile.py
[linux_x86_64]
[linux_x86_64] Downloading conan_export.tgz
[linux_x86_64]
[linux_x86_64] ....
[linux_x86_64] libmagic/5.33@jenkins/stable: Not found in local cache, looking in remotes...
[linux_x86_64] libmagic/5.33@jenkins/stable: Trying with 'conan-local'...
[linux_x86_64] Downloading conanmanifest.txt
[linux_x86_64]
[linux_x86_64] Downloading conanfile.py
[linux_x86_64]
[linux_x86_64] ERROR: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
[linux_x86_64]
[linux_x86_64] Unable to connect to conan-center=https://conan.bintray.com
[linux_x86_64] /usr/local/lib/python3.5/site-packages/urllib3/util/ssl_.py:352: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
[linux_x86_64]   SNIMissingWarning
```

So, I believe in the case of multiple remotes, with conan 1.9 both remotes are contacted when a package is downloaded (but not when a package is built). The build slave is happy if I pin conan to 1.8.4.


Discussion:
It looks like it is trying to search for the binary in the second remote without following the remote used to download the recipe. I will try to confirm it tomorrow.

As it is a different behaviour from 1.8 I assume that it should be fixed, but this flow have some benefits like having different binaries (maybe for different architectures) in a second repository, for example I have a personal repository with ARM packages for some Conan-center packages. A simple install will resolve all the binaries.

In your case, shouldn't be better for you to specify the remote with "-r" or removing Conan center remote if it shouldn't be contacted?
I agree I can remove conan-center, and probably will. Figured I would raise it just in case it was unexpected.

I didn't think of the ARM usecase you mentioned above, how do you ensure packages are in sync across multiple repo's? How do you resolve differences between the conanfiles/exports etc. between the repos (assuming same version)?
Very good point, the answer is in the package revisions #3055 feature. Still a work in progress but Conan will be progressively introducing that feature until Conan 2.0 is released.

With package revisions a recipe is unique and you could have different binaries for the same recipe revision in different remotes. You could safety mix them because you can guarantee that the used recipe to generate the binaries is the same.

So when revisions feature is fully deployed the mixing of binaries from different remotes will be a valid flow.

Until then, I agree we need to revert the behaviour to the previous one (1.8.4)
I tried to reproduce the issue without success. Are you using `config install` to install a `registry.txt` file maybe?
Could you send me the contents of the `registry.json` file in your slave? It could be that for some reason, I still don't know, in the registry file the reference for the `libmagic` is pointing to the bintray remote.

Resolving patch:
diff --git a/conans/client/graph/graph_binaries.py b/conans/client/graph/graph_binaries.py
--- a/conans/client/graph/graph_binaries.py
+++ b/conans/client/graph/graph_binaries.py
@@ -81,7 +81,10 @@ def _evaluate_node(self, node, build_mode, update, evaluated_references, remote_
         if remote_name:
             remote = self._registry.remotes.get(remote_name)
         else:
-            remote = self._registry.prefs.get(package_ref)
+            # If the remote_name is not given, follow the binary remote, or
+            # the recipe remote
+            # If it is defined it won't iterate (might change in conan2.0)
+            remote = self._registry.prefs.get(package_ref) or self._registry.refs.get(conan_ref)
         remotes = self._registry.remotes.list
 
         if os.path.exists(package_folder):
diff --git a/conans/client/remover.py b/conans/client/remover.py
--- a/conans/client/remover.py
+++ b/conans/client/remover.py
@@ -114,6 +114,7 @@ def _local_remove(self, reference, src, build_ids, package_ids):
         if not src and build_ids is None and package_ids is None:
             remover.remove(reference)
             self._registry.refs.remove(reference, quiet=True)
+            self._registry.prefs.remove_all(reference)
 
     def remove(self, pattern, remote_name, src=None, build_ids=None, package_ids_filter=None, force=False,
                packages_query=None, outdated=False):


Test patch:
diff --git a/conans/test/command/remove_test.py b/conans/test/command/remove_test.py
--- a/conans/test/command/remove_test.py
+++ b/conans/test/command/remove_test.py
@@ -12,6 +12,27 @@
 from conans.test.utils.tools import TestClient, TestBufferConanOutput, TestServer
 from conans.test.utils.cpp_test_files import cpp_hello_conan_files
 from conans.test.utils.test_files import temp_folder
+from conans.util.files import load
+
+
+class RemoveRegistryTest(unittest.TestCase):
+
+    def remove_registry_test(self):
+        test_server = TestServer(users={"lasote": "password"})  # exported users and passwords
+        servers = {"default": test_server}
+        client = TestClient(servers=servers, users={"default": [("lasote", "password")]})
+        conanfile = """from conans import ConanFile
+class Test(ConanFile):
+    pass
+    """
+        client.save({"conanfile.py": conanfile})
+        client.run("create . Test/0.1@lasote/testing")
+        client.run("upload * --all --confirm")
+        client.run('remove "*" -f')
+        client.run("remote list_pref Test/0.1@lasote/testing")
+        self.assertNotIn("Test/0.1@lasote/testing", client.out)
+        registry_content = load(client.client_cache.registry)
+        self.assertNotIn("Test/0.1@lasote/testing", registry_content)
 
 
 class RemoveOutdatedTest(unittest.TestCase):
diff --git a/conans/test/integration/multi_remote_test.py b/conans/test/integration/multi_remote_test.py
--- a/conans/test/integration/multi_remote_test.py
+++ b/conans/test/integration/multi_remote_test.py
@@ -1,9 +1,13 @@
 import unittest
-from conans.test.utils.tools import TestServer, TestClient
-from conans.model.ref import ConanFileReference, PackageReference
-from conans.test.utils.cpp_test_files import cpp_hello_conan_files
 from collections import OrderedDict
 
+from conans.model.ref import ConanFileReference
+from conans.test.utils.cpp_test_files import cpp_hello_conan_files
+from conans.test.utils.tools import TestServer, TestClient
+from conans.util.files import load
+import json
+import time
+
 
 class MultiRemoteTest(unittest.TestCase):
 
@@ -115,39 +119,47 @@ def install_from_remotes_test(self):
         self.assertIn("Remote: remote1=http://", client2.user_io.out)
         self.assertIn("Remote: remote2=http://", client2.user_io.out)
 
-    def package_remote_follows_recipe_remote_test(self):
+    def package_binary_remote_test(self):
+        # https://github.com/conan-io/conan/issues/3882
         conanfile = """from conans import ConanFile
 class ConanFileToolsTest(ConanFile):
     pass
 """
-
         # Upload recipe + package to remote1 and remote2
         ref = "Hello/0.1@lasote/stable"
         self.client.save({"conanfile.py": conanfile})
         self.client.run("create . %s" % ref)
-        self.client.run("upload %s -r=remote1 --all" % ref)
+        self.client.run("upload %s -r=remote0 --all" % ref)
         self.client.run("upload %s -r=remote2 --all" % ref)
 
         # Remove only binary from remote1 and everything in local
-        self.client.run("remove -f %s -p5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 -r remote1" % ref)
-        self.client.run('remove %s -f' % ref)
+        self.client.run("remove -f %s -p -r remote0" % ref)
+        self.client.run('remove "*" -f')
 
+        self.servers.pop("remote1")
         # Now install it from a client, it won't find the binary in remote2
         error = self.client.run("install %s" % ref, ignore_error=True)
         self.assertTrue(error)
         self.assertIn("Can't find a 'Hello/0.1@lasote/stable' package", self.client.out)
         self.assertNotIn("remote2", self.client.out)
 
-        self.client.run("install %s -r remote2" % ref, ignore_error=False)
+        self.client.run("install %s -r remote2" % ref)
         self.assertIn("Package installed 5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9", self.client.out)
-
-        ref = ConanFileReference.loads(ref)
-        pref = PackageReference(ref, "5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9")
-
-        # Package from remote2
-        remote = self.client.remote_registry.prefs.get(pref)
-        self.assertEquals(remote.name, "remote2")
-
-        # But recipe from remote1
-        remote = self.client.remote_registry.refs.get(ref)
-        self.assertEquals(remote.name, "remote1")
+        registry = load(self.client.client_cache.registry)
+        registry = json.loads(registry)
+        self.assertEquals(registry["references"], {"Hello/0.1@lasote/stable": "remote0"})
+        self.assertEquals(registry["package_references"],
+                          {"Hello/0.1@lasote/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9":
+                           "remote2"})
+
+        client = TestClient(servers=self.servers, users=self.users)
+        time.sleep(1)  # Make sure timestamps increase
+        client.save({"conanfile.py": conanfile + " # Comment"})
+        client.run("create . %s" % ref)
+        client.run("upload %s -r=remote2 --all" % ref)
+        self.client.run("install %s --update" % ref)
+        self.assertNotIn("Hello/0.1@lasote/stable: WARN: Can't update, no package in remote",
+                         self.client.out)
+        self.assertIn("Hello/0.1@lasote/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Update",
+                      self.client.out)
+        self.assertIn("Downloading conan_package.tgz", self.client.out)