Repository: conan-io/conan

Issue:
[bug] passwords with special characters not properly encoded when checking out sources from SCM
Authentication started failing after upgrading to 1.32.0

### Environment Details (include every applicable attribute)
  * Operating System+version:
  * Compiler+version:
  * Conan version: 1.32.0
  * Python version:

### Logs (Executed commands with output) (Include/Attach if Applicable)
command:
```
conan install my_package/1.0.0@user/channel#f6c6506327598117fef706344bbf6bb11bcb8f3c --build=my_package/1.0.0@user/channel#f6c6506327598117fef706344bbf6bb11bcb8f3c --build=missing --lockfile=conan.lock --lockfile-out=updated_conan.lock 
```
output:
```
my_package/1.0.0@user/channel: SCM: Getting sources from url: 'https://my/repo/'
ERROR: Couldn't checkout SCM: Command 'git -c http.sslVerify=true clone "https://****:****@my/repo/" .   ' returned non-zero exit status 128.
Cloning into '.'...
remote: Unauthorized
fatal: Authentication failed for 'https://my/repo/'
```



Discussion:
This could come from https://github.com/conan-io/conan/pull/8016
I am trying to add a unittest with special characters, like:

```python
    def test_url_with_user_password_characters_param(self):
        scm = SCMBase(username="user", password="p.-+*¨Ç{}%&$()9")
        self.assertEqual('https://user:p.-+*¨Ç{}%&$()9@github.com/conan-io/conan.git',
                         scm.get_url_with_credentials("https://github.com/conan-io/conan.git"))
```
But this works fine. Can you please detail how are you declaring your scm, and how are you passing the password?
Hi @memsharded,

I'm passing the username and password through environmental variables like so:

_conanfile.py_

```
    scm = {
        "type": "git",
        "url": "auto",
        "revision": "auto",
        "username": tools.get_env("GIT_USER"),
        "password": tools.get_env("GIT_PASS")
    }
```

_Jenkinsfile_
```
withCredentials ([usernamePassword(credentialsId: 'jenkins_git_creds', \
                                   passwordVariable: 'GIT_PASS', \
                                   usernameVariable: 'GIT_USER')]) {
    command (
        label: "Build ${ref}",
        script: "conan install ${ref} --build=${ref} --build=missing --lockfile=${lockfile} --lockfile-out=${updated_lockfile}"
    )
 }
```
Just a quick update.  

Before 1.32.0, the password was provided url-encoded (e.g. `%` to `%25`) and with OS special characters escaped. (e.g. `%` to `%%`)

This breaks in 1.32.0.  It seems removing the OS escapes solves the issue.

Can you please advise how passwords should be provided?

I would still consider this to be a breaking change since CI scripts must change their behavior depending on installed conan version on the build machine.

Resolving patch:
diff --git a/conans/client/tools/scm.py b/conans/client/tools/scm.py
--- a/conans/client/tools/scm.py
+++ b/conans/client/tools/scm.py
@@ -69,27 +69,38 @@ def _handle_scp_pattern(self, user, domain, url):
         return "{user}@{domain}:{url}".format(user=user, domain=domain, url=url)
 
     def _handle_url_pattern(self, scheme, url, user=None, password=None):
-        if scheme == "file":
+        if scheme in ["file", "git"]:
             if self._username:
-                self._output.warn("SCM username cannot be set for file url, ignoring parameter")
+                self._output.warn("SCM username cannot be set for {} url, ignoring "
+                                  "parameter".format(scheme))
             if self._password:
-                self._output.warn("SCM password cannot be set for file url, ignoring parameter")
+                self._output.warn("SCM password cannot be set for {} url, ignoring "
+                                  "parameter".format(scheme))
+            if user or password:
+                self._output.warn("Username/Password in URL cannot be set for '{}' SCM type, "
+                                  "removing it".format(scheme))
             return "{scheme}://{url}".format(scheme=scheme, url=url)
         elif scheme == "ssh" and self._password:
             self._output.warn("SCM password cannot be set for ssh url, ignoring parameter")
+        elif password and self._password:
+            self._output.warn("SCM password got from URL, ignoring 'password' parameter")
 
         if user and self._username:
             self._output.warn("SCM username got from URL, ignoring 'username' parameter")
-        if password and self._password:
-            self._output.warn("SCM password got from URL, ignoring 'password' parameter")
 
         the_user = user or self._username
         the_password = password or self._password
+
         if the_password and the_user and scheme != "ssh":
-            return "{scheme}://{user}:{password}@{url}".format(scheme=scheme, user=the_user,
-                                                               password=the_password, url=url)
+            return "{scheme}://{user}:{password}@{url}".format(scheme=scheme,
+                                                               user=quote_plus(the_user),
+                                                               password=quote_plus(the_password),
+                                                               url=url)
         elif the_user:
-            return "{scheme}://{user}@{url}".format(scheme=scheme, user=the_user, url=url)
+            if scheme == "ssh" and password:
+                self._output.warn("Password in URL cannot be set for 'ssh' SCM type, removing it")
+            return "{scheme}://{user}@{url}".format(scheme=scheme, user=quote_plus(the_user),
+                                                    url=url)
         else:
             return "{scheme}://{url}".format(scheme=scheme, url=url)
 
@@ -98,8 +109,8 @@ def get_url_with_credentials(self, url):
             return url
 
         scp_regex = re.compile("^(?P<user>[a-zA-Z0-9_]+)@(?P<domain>[a-zA-Z0-9._-]+):(?P<url>.*)$")
-        url_user_pass_regex = re.compile("^(?P<scheme>file|http|https|git):\/\/(?P<user>\S+):(?P<password>\S+)@(?P<url>.*)$")
-        url_user_regex = re.compile("^(?P<scheme>file|http|https|git|ssh):\/\/(?P<user>\S+)@(?P<url>.*)$")
+        url_user_pass_regex = re.compile("^(?P<scheme>file|http|https|git|ssh):\/\/(?P<user>\w+):(?P<password>\w+)@(?P<url>.*)$")
+        url_user_regex = re.compile("^(?P<scheme>file|http|https|git|ssh):\/\/(?P<user>\w+)@(?P<url>.*)$")
         url_basic_regex = re.compile("^(?P<scheme>file|http|https|git|ssh):\/\/(?P<url>.*)$")
 
         url_patterns = [


Test patch:
diff --git a/conans/test/unittests/client/tools/scm/test_scm_base.py b/conans/test/unittests/client/tools/scm/test_scm_base.py
--- a/conans/test/unittests/client/tools/scm/test_scm_base.py
+++ b/conans/test/unittests/client/tools/scm/test_scm_base.py
@@ -102,6 +102,11 @@ def test_url_with_user_password_param(self):
         self.assertEqual('https://user:pass@github.com/conan-io/conan.git',
                          scm.get_url_with_credentials("https://github.com/conan-io/conan.git"))
 
+    def test_url_with_user_password_characters_param(self):
+        scm = SCMBase(username="el niño", password="la contra%seña")
+        self.assertEqual('https://el+ni%C3%B1o:la+contra%25se%C3%B1a@github.com/conan-io/conan.git',
+                         scm.get_url_with_credentials("https://github.com/conan-io/conan.git"))
+
     def test_url_user_with_user_param(self):
         output = OutputMock()
         scm = SCMBase(username="user", output=output)
@@ -204,28 +209,33 @@ def test_ssh_url_with_username_and_username_password(self):
     def test_ssh_url_with_username_password_and_only_password(self):
         output = OutputMock()
         scm = SCMBase(password="password", output=output)
-        self.assertEqual('ssh://git:pass@github.com/conan-io/conan.git',
+        self.assertEqual('ssh://git@github.com/conan-io/conan.git',
                          scm.get_url_with_credentials("ssh://git:pass@github.com/conan-io/conan.git"))
-        self.assertEqual(1, len(output.out))
+        self.assertEqual(2, len(output.out))
         self.assertIn("WARN: SCM password cannot be set for ssh url, ignoring parameter", output.out)
+        self.assertIn("WARN: Password in URL cannot be set for 'ssh' SCM type, removing it",
+                      output.out)
 
     def test_ssh_url_with_username_password_and_only_username(self):
         output = OutputMock()
         scm = SCMBase(username="dani", output=output)
-        self.assertEqual('ssh://git:pass@github.com/conan-io/conan.git',
+        self.assertEqual('ssh://git@github.com/conan-io/conan.git',
                          scm.get_url_with_credentials("ssh://git:pass@github.com/conan-io/conan.git"))
-        self.assertEqual(1, len(output.out))
-        self.assertIn("WARN: SCM username got from URL, ignoring 'username' parameter",
+        self.assertEqual(2, len(output.out))
+        self.assertIn("WARN: SCM username got from URL, ignoring 'username' parameter", output.out)
+        self.assertIn("WARN: Password in URL cannot be set for 'ssh' SCM type, removing it",
                       output.out)
 
     def test_ssh_url_with_username_password_and_username_password(self):
         output = OutputMock()
         scm = SCMBase(password="password", username="dani", output=output)
-        self.assertEqual("ssh://git:pass@github.com/conan-io/conan.git",
+        self.assertEqual("ssh://git@github.com/conan-io/conan.git",
                          scm.get_url_with_credentials("ssh://git:pass@github.com/conan-io/conan.git"))
-        self.assertEqual(2, len(output.out))
+        self.assertEqual(3, len(output.out))
         self.assertIn("WARN: SCM password cannot be set for ssh url, ignoring parameter", output.out)
         self.assertIn("WARN: SCM username got from URL, ignoring 'username' parameter", output.out)
+        self.assertIn("WARN: Password in URL cannot be set for 'ssh' SCM type, removing it",
+                      output.out)
 
     def test_scp(self):
         scm = SCMBase()
@@ -276,3 +286,42 @@ def test_file_url_with_username_password_params(self):
                       output.out)
         self.assertIn("WARN: SCM password cannot be set for file url, ignoring parameter",
                       output.out)
+
+    def test_git(self):
+        scm = SCMBase()
+        self.assertEqual('git://github.com/conan-io/conan.git',
+                         scm.get_url_with_credentials("git://github.com/conan-io/conan.git"))
+
+    def test_git_only_password(self):
+        output = OutputMock()
+        scm = SCMBase(password="pass", output=output)
+        self.assertEqual("git://github.com/conan-io/conan.git",
+                         scm.get_url_with_credentials("git://github.com/conan-io/conan.git"))
+        self.assertIn("WARN: SCM password cannot be set for git url, ignoring parameter", output.out)
+
+    def test_git_only_username(self):
+        output = OutputMock()
+        scm = SCMBase(username="dani", output=output)
+        self.assertEqual("git://github.com/conan-io/conan.git",
+                         scm.get_url_with_credentials("git://github.com/conan-io/conan.git"))
+        self.assertIn("WARN: SCM username cannot be set for git url, ignoring parameter", output.out)
+
+    def test_git_username_password(self):
+        output = OutputMock()
+        scm = SCMBase(password="pass", username="dani", output=output)
+        self.assertEqual("git://github.com/conan-io/conan.git",
+                         scm.get_url_with_credentials("git://github.com/conan-io/conan.git"))
+        self.assertEqual(2, len(output.out))
+        self.assertIn("WARN: SCM password cannot be set for git url, ignoring parameter", output.out)
+        self.assertIn("WARN: SCM password cannot be set for git url, ignoring parameter", output.out)
+
+    def test_git_url_username_password(self):
+        output = OutputMock()
+        scm = SCMBase(password="pass", output=output)
+        self.assertEqual("git://github.com/conan-io/conan.git",
+                         scm.get_url_with_credentials(
+                             "git://user:pass@github.com/conan-io/conan.git"))
+        self.assertEqual(2, len(output.out))
+        self.assertIn("WARN: SCM password cannot be set for git url, ignoring parameter", output.out)
+        self.assertIn("WARN: Username/Password in URL cannot be set for 'git' SCM type, removing it",
+                      output.out)