Repository: buildbot/buildbot

Issue:
Unmodified builder name used in ssh path
Buildbot version: 2.6.0

I recently discovered that while a Unix worker will work very well with builder names having almost all sorts of characters, such as `:`, a Windows worker has issues.

The documentation for builder builddirs does specify that ["this parameter defaults to the builder name, with some characters escaped"](http://docs.buildbot.net/latest/manual/configuration/builders.html) (in practiced, that seems to mean that certain characters are changed to underscores).  However, it doesn't like the same type of munging doesn't happen for the SSH path, and this function seems to confirm that:

https://github.com/buildbot/buildbot/blob/9f185536ff1138e125a51fe8c028334fb9b4356d/master/buildbot/util/git.py#L147-L170

(I checked, this function hasn't changed between v2.6.0 and now)

What happens on Windows is that the git setup step fails, like this (`mkdir` is the problem...  the `stat` line is to show the builder name munging I was expecting to happen in *any* path):

```
stat: The system cannot find the file specified: C:\Users\buildworker\vc-win32\builder_name_with_colons\build\.buildbot-patchedprogram finished with exit code 2
mkdir: The filename, directory name, or volume label syntax is incorrect: C:\Users\buildworker\vc-win32\.builder:name:with:colons.build.buildbotprogram finished with exit code 22
```


Discussion:
Good finding! I think you are only a few steps away to fixing this issue..

Would you like to submit a patch?

I can give it a shot, given my weak python-fu (I'm pretty much a beginning learner)
Wait, hold on....   naaaaahhhhh, can't be *that* simple, can it?
Well, ok, if that was it, PR comming up!

Resolving patch:
diff --git a/master/buildbot/util/git.py b/master/buildbot/util/git.py
--- a/master/buildbot/util/git.py
+++ b/master/buildbot/util/git.py
@@ -23,6 +23,7 @@
 from buildbot.process import buildstep
 from buildbot.process import remotecommand
 from buildbot.process.properties import Properties
+from buildbot.util import bytes2unicode
 
 RC_SUCCESS = 0
 
@@ -150,13 +151,15 @@ def _getSshDataPath(self):
         # destination directory being non-empty. We have to use separate
         # temporary directory for that data to ensure the confidentiality of it.
         # So instead of
-        # '{path}/{to}/{workdir}/.buildbot-ssh-key' we put the key at
-        # '{path}/{to}/.{builder_name}.{workdir}.buildbot/ssh-key'.
+        # '{path}/{to}/{workerbuilddir}/{workdir}/.buildbot-ssh-key'
+        # we put the key in
+        # '{path}/{to}/.{workerbuilddir}.{workdir}.buildbot/ssh-key'.
 
         # basename and dirname interpret the last element being empty for paths
         # ending with a slash
         path_module = self.build.path_module
 
+        workerbuilddir = bytes2unicode(self.build.builder.config.workerbuilddir)
         workdir = self._getSshDataWorkDir().rstrip('/\\')
 
         if path_module.isabs(workdir):
@@ -165,7 +168,7 @@ def _getSshDataPath(self):
             parent_path = path_module.join(self.worker.worker_basedir,
                                            path_module.dirname(workdir))
 
-        basename = '.{0}.{1}.buildbot'.format(self.build.builder.name,
+        basename = '.{0}.{1}.buildbot'.format(workerbuilddir,
                                               path_module.basename(workdir))
         return path_module.join(parent_path, basename)
 


Test patch:
diff --git a/master/buildbot/test/unit/steps/test_source_git.py b/master/buildbot/test/unit/steps/test_source_git.py
--- a/master/buildbot/test/unit/steps/test_source_git.py
+++ b/master/buildbot/test/unit/steps/test_source_git.py
@@ -136,8 +136,8 @@ def test_mode_full_clean_ssh_key_2_10(self):
             self.stepClass(repourl='http://github.com/buildbot/buildbot.git',
                            mode='full', method='clean', sshPrivateKey='sshkey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
         ssh_command_config = \
             'core.sshCommand=ssh -o "BatchMode=yes" -i "{0}"'.format(ssh_key_path)
 
@@ -195,8 +195,8 @@ def test_mode_full_clean_ssh_key_2_3(self):
             self.stepClass(repourl='http://github.com/buildbot/buildbot.git',
                            mode='full', method='clean', sshPrivateKey='sshkey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
         ssh_command = 'ssh -o "BatchMode=yes" -i "{0}"'.format(ssh_key_path)
 
         self.expectCommands(
@@ -254,9 +254,9 @@ def test_mode_full_clean_ssh_key_1_7(self):
             self.stepClass(repourl='http://github.com/buildbot/buildbot.git',
                            mode='full', method='clean', sshPrivateKey='sshkey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
-        ssh_wrapper_path = '/wrk/.Builder.wkdir.buildbot/ssh-wrapper.sh'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
+        ssh_wrapper_path = '/wrk/.bldr.wkdir.buildbot/ssh-wrapper.sh'
 
         # A place to store what gets read
         read = []
@@ -330,9 +330,9 @@ def test_mode_full_clean_ssh_host_key_2_10(self, name, class_params):
             self.stepClass(repourl='http://github.com/buildbot/buildbot.git',
                            mode='full', method='clean', sshPrivateKey='sshkey', **class_params))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
-        ssh_known_hosts_path = '/wrk/.Builder.wkdir.buildbot/ssh-known-hosts'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
+        ssh_known_hosts_path = '/wrk/.bldr.wkdir.buildbot/ssh-known-hosts'
         ssh_command_config = \
             'core.sshCommand=ssh -o "BatchMode=yes" -i "{0}" ' \
             '-o "UserKnownHostsFile={1}"'.format(ssh_key_path,
@@ -400,9 +400,9 @@ def test_mode_full_clean_ssh_host_key_2_3(self):
                            mode='full', method='clean', sshPrivateKey='sshkey',
                            sshHostKey='sshhostkey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
-        ssh_known_hosts_path = '/wrk/.Builder.wkdir.buildbot/ssh-known-hosts'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
+        ssh_known_hosts_path = '/wrk/.bldr.wkdir.buildbot/ssh-known-hosts'
         ssh_command = \
             'ssh -o "BatchMode=yes" -i "{0}" ' \
             '-o "UserKnownHostsFile={1}"'.format(ssh_key_path,
@@ -471,10 +471,10 @@ def test_mode_full_clean_ssh_host_key_1_7(self):
                            mode='full', method='clean', sshPrivateKey='sshkey',
                            sshHostKey='sshhostkey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
-        ssh_wrapper_path = '/wrk/.Builder.wkdir.buildbot/ssh-wrapper.sh'
-        ssh_known_hosts_path = '/wrk/.Builder.wkdir.buildbot/ssh-known-hosts'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
+        ssh_wrapper_path = '/wrk/.bldr.wkdir.buildbot/ssh-wrapper.sh'
+        ssh_known_hosts_path = '/wrk/.bldr.wkdir.buildbot/ssh-known-hosts'
 
         # A place to store what gets read
         read = []
@@ -555,10 +555,10 @@ def test_mode_full_clean_ssh_host_key_1_7_progress(self):
                            mode='full', method='clean', sshPrivateKey='sshkey',
                            sshHostKey='sshhostkey', progress=True))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
-        ssh_wrapper_path = '/wrk/.Builder.wkdir.buildbot/ssh-wrapper.sh'
-        ssh_known_hosts_path = '/wrk/.Builder.wkdir.buildbot/ssh-known-hosts'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
+        ssh_wrapper_path = '/wrk/.bldr.wkdir.buildbot/ssh-wrapper.sh'
+        ssh_known_hosts_path = '/wrk/.bldr.wkdir.buildbot/ssh-known-hosts'
 
         self.expectCommands(
             ExpectShell(workdir='wkdir',
@@ -632,9 +632,9 @@ def test_mode_full_clean_ssh_host_key_2_10_abs_workdir(self):
         workdir = '/myworkdir/workdir'
         self.build.workdir = workdir
 
-        ssh_workdir = '/myworkdir/.Builder.workdir.buildbot'
-        ssh_key_path = '/myworkdir/.Builder.workdir.buildbot/ssh-key'
-        ssh_known_hosts_path = '/myworkdir/.Builder.workdir.buildbot/ssh-known-hosts'
+        ssh_workdir = '/myworkdir/.bldr.workdir.buildbot'
+        ssh_key_path = '/myworkdir/.bldr.workdir.buildbot/ssh-key'
+        ssh_known_hosts_path = '/myworkdir/.bldr.workdir.buildbot/ssh-known-hosts'
         ssh_command_config = \
             'core.sshCommand=ssh -o "BatchMode=yes" -i "{0}" ' \
             '-o "UserKnownHostsFile={1}"'.format(ssh_key_path,
@@ -742,8 +742,8 @@ def test_mode_full_clean_win32path_ssh_key_2_10(self):
                            mode='full', method='clean', sshPrivateKey='sshkey'))
         self.changeWorkerSystem('win32')
 
-        ssh_workdir = '\\wrk\\.Builder.wkdir.buildbot'
-        ssh_key_path = '\\wrk\\.Builder.wkdir.buildbot\\ssh-key'
+        ssh_workdir = '\\wrk\\.bldr.wkdir.buildbot'
+        ssh_key_path = '\\wrk\\.bldr.wkdir.buildbot\\ssh-key'
         ssh_command_config = 'core.sshCommand=ssh -o "BatchMode=yes" -i "{0}"'.format(ssh_key_path)
 
         self.expectCommands(
@@ -801,8 +801,8 @@ def test_mode_full_clean_win32path_ssh_key_2_3(self):
                            mode='full', method='clean', sshPrivateKey='sshkey'))
         self.changeWorkerSystem('win32')
 
-        ssh_workdir = '\\wrk\\.Builder.wkdir.buildbot'
-        ssh_key_path = '\\wrk\\.Builder.wkdir.buildbot\\ssh-key'
+        ssh_workdir = '\\wrk\\.bldr.wkdir.buildbot'
+        ssh_key_path = '\\wrk\\.bldr.wkdir.buildbot\\ssh-key'
         ssh_command = 'ssh -o "BatchMode=yes" -i "{0}"'.format(ssh_key_path)
 
         self.expectCommands(
@@ -860,9 +860,9 @@ def test_mode_full_clean_win32path_ssh_key_1_7(self):
                            mode='full', method='clean', sshPrivateKey='sshkey'))
         self.changeWorkerSystem('win32')
 
-        ssh_workdir = '\\wrk\\.Builder.wkdir.buildbot'
-        ssh_key_path = '\\wrk\\.Builder.wkdir.buildbot\\ssh-key'
-        ssh_wrapper_path = '\\wrk\\.Builder.wkdir.buildbot\\ssh-wrapper.sh'
+        ssh_workdir = '\\wrk\\.bldr.wkdir.buildbot'
+        ssh_key_path = '\\wrk\\.bldr.wkdir.buildbot\\ssh-key'
+        ssh_wrapper_path = '\\wrk\\.bldr.wkdir.buildbot\\ssh-wrapper.sh'
 
         self.expectCommands(
             ExpectShell(workdir='wkdir',
@@ -1867,8 +1867,8 @@ def test_mode_incremental_branch_ssh_key_2_10(self):
                            mode='incremental', branch='test-branch',
                            sshPrivateKey='ssh-key', progress=True))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
         ssh_command_config = \
             'core.sshCommand=ssh -o "BatchMode=yes" -i "{0}"'.format(ssh_key_path)
 
@@ -2572,8 +2572,8 @@ def test_mode_full_copy_ssh_key_2_10(self):
             self.stepClass(repourl='http://github.com/buildbot/buildbot.git',
                            mode='full', method='copy', sshPrivateKey='sshkey'))
 
-        ssh_workdir = '/wrk/.Builder.source.buildbot'
-        ssh_key_path = '/wrk/.Builder.source.buildbot/ssh-key'
+        ssh_workdir = '/wrk/.bldr.source.buildbot'
+        ssh_key_path = '/wrk/.bldr.source.buildbot/ssh-key'
         ssh_command_config = \
             'core.sshCommand=ssh -o "BatchMode=yes" -i "{0}"'.format(ssh_key_path)
 
@@ -3680,8 +3680,8 @@ def test_push_ssh_key_2_10(self):
             self.stepClass(workdir='wkdir', repourl=url,
                            branch='testbranch', sshPrivateKey='sshKey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
         ssh_command_config = \
             'core.sshCommand=ssh -o "BatchMode=yes" -i "{0}"'.format(ssh_key_path)
 
@@ -3720,8 +3720,8 @@ def test_push_ssh_key_2_3(self):
             self.stepClass(workdir='wkdir', repourl=url,
                            branch='testbranch', sshPrivateKey='sshKey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
         ssh_command = 'ssh -o "BatchMode=yes" -i "{0}"'.format(ssh_key_path)
 
         self.expectCommands(
@@ -3759,9 +3759,9 @@ def test_push_ssh_key_1_7(self):
             self.stepClass(workdir='wkdir', repourl=url,
                            branch='testbranch', sshPrivateKey='sshKey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
-        ssh_wrapper_path = '/wrk/.Builder.wkdir.buildbot/ssh-wrapper.sh'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
+        ssh_wrapper_path = '/wrk/.bldr.wkdir.buildbot/ssh-wrapper.sh'
 
         self.expectCommands(
             ExpectShell(workdir='wkdir',
@@ -3806,9 +3806,9 @@ def test_push_ssh_host_key_2_10(self):
                            branch='testbranch', sshPrivateKey='sshkey',
                            sshHostKey='sshhostkey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
-        ssh_known_hosts_path = '/wrk/.Builder.wkdir.buildbot/ssh-known-hosts'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
+        ssh_known_hosts_path = '/wrk/.bldr.wkdir.buildbot/ssh-known-hosts'
         ssh_command_config = \
             'core.sshCommand=ssh -o "BatchMode=yes" -i "{0}" ' \
             '-o "UserKnownHostsFile={1}"'.format(ssh_key_path,
@@ -3856,9 +3856,9 @@ def test_push_ssh_host_key_2_3(self):
                            branch='testbranch', sshPrivateKey='sshkey',
                            sshHostKey='sshhostkey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
-        ssh_known_hosts_path = '/wrk/.Builder.wkdir.buildbot/ssh-known-hosts'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
+        ssh_known_hosts_path = '/wrk/.bldr.wkdir.buildbot/ssh-known-hosts'
         ssh_command = \
             'ssh -o "BatchMode=yes" -i "{0}" ' \
             '-o "UserKnownHostsFile={1}"'.format(ssh_key_path,
@@ -3906,10 +3906,10 @@ def test_push_ssh_host_key_1_7(self):
                            branch='testbranch', sshPrivateKey='sshkey',
                            sshHostKey='sshhostkey'))
 
-        ssh_workdir = '/wrk/.Builder.wkdir.buildbot'
-        ssh_key_path = '/wrk/.Builder.wkdir.buildbot/ssh-key'
-        ssh_wrapper_path = '/wrk/.Builder.wkdir.buildbot/ssh-wrapper.sh'
-        ssh_known_hosts_path = '/wrk/.Builder.wkdir.buildbot/ssh-known-hosts'
+        ssh_workdir = '/wrk/.bldr.wkdir.buildbot'
+        ssh_key_path = '/wrk/.bldr.wkdir.buildbot/ssh-key'
+        ssh_wrapper_path = '/wrk/.bldr.wkdir.buildbot/ssh-wrapper.sh'
+        ssh_known_hosts_path = '/wrk/.bldr.wkdir.buildbot/ssh-known-hosts'
 
         self.expectCommands(
             ExpectShell(workdir='wkdir',