Repository: Gallopsled/pwntools

Issue:
python3: tubes.ssh.run() - env(dict) - binary literals and strings in values are encoded incorrectly
Hi,

I tried to do write automagical solver of some wargame with use of pwntools. I got the problem when it comes to set environment variable for execution on remote host via ssh by system() method (aka run()).

As to not disclose the challange for others, I prepared some examplary problem instead of mentioning wargame itself. If you want to know it exactly ask me on my loginname on Gmail Dot Com.

Bear in mind that maybe I miss some python skill so sorry if You find it invalid. I also open to fix it and test.

Here it goes. In local directory on `builder@localhost` there is small shell script named echo.sh which is very simple and look for environment variable.
```
#!/bin/bash

env | grep varia
```

There we go with trying to get some bytes back.
```
Python 3.8.1 (default, Jan 22 2020, 06:38:00) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.12.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from pwn import *                                                                                                                                    

In [2]: pwnlib.__version__                                                                                                                                   
Out[2]: '4.2.0dev'

In [3]: _ssh = ssh(host='localhost',user='builder')                                                                                                          
[x] Connecting to localhost on port 22
[+] Connecting to localhost on port 22: Done
[*] builder@localhost:
    Distro    Unknown 
    OS:       linux
    Arch:     amd64
    Version:  4.19.101
    ASLR:     Enabled

In [4]: _run1 = _ssh.run(b'varia=asdf ./echo.sh')                                                                                                            
[x] Opening new channel: b'varia=asdf ./echo.sh'
[+] Opening new channel: b'varia=asdf ./echo.sh': Done

In [5]: _run1.recvall()                                                                                                                                      
[x] Receiving all data
[x] Receiving all data: 0B
[x] Receiving all data: 11B
[+] Receiving all data: Done (11B)
[*] Closed SSH channel with localhost
Out[5]: b'varia=asdf\n'

In [6]: _run2 = _ssh.run(b'./echo.sh',env={'varia':b'asdf'})                                                                                                 
[x] Opening new channel: b'./echo.sh'
[+] Opening new channel: b'./echo.sh': Done

In [7]: _run2.recvall()                                                                                                                                      
[x] Receiving all data
[x] Receiving all data: 0B
[x] Receiving all data: 12B
[+] Receiving all data: Done (12B)
[*] Closed SSH channel with localhost
Out[7]: b'varia=basdf\n'

In [8]: _run3 = _ssh.run(b'./echo.sh',env={'varia':'asdf'})                                                                                                  
[x] Opening new channel: b'./echo.sh'
[+] Opening new channel: b'./echo.sh': Done

In [9]: _run3.recvall()                                                                                                                                      
[x] Receiving all data
[x] Receiving all data: 0B
[x] Receiving all data: 11B
[+] Receiving all data: Done (11B)
[*] Closed SSH channel with localhost
Out[9]: b'varia=asdf\n'

```

![ipython_output](https://user-images.githubusercontent.com/61562700/75494947-a7600400-59bd-11ea-858a-d580244749c9.png)

Here, we see that additional *b* letter was added in case of binary litteral.

Bigger problem is when we provide the binary literal or string literal to it and one got's *b* letter and apostrophes, but second got 'latin1' encoding (same output with str(bytes,'latin1') ).

```
Python 3.8.1 (default, Jan 22 2020, 06:38:00) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.12.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from pwn import *                                                                            

In [2]: _ssh = ssh(host='localhost',user='builder')                                                  
[x] Connecting to localhost on port 22
[+] Connecting to localhost on port 22: Done
[*] builder@localhost:
    Distro    Unknown 
    OS:       linux
    Arch:     amd64
    Version:  4.19.101
    ASLR:     Enabled

In [3]: _run1 = _ssh.run(b'varia=asdf\x90 ./echo.sh')                                                
[x] Opening new channel: b'varia=asdf\x90 ./echo.sh'
[+] Opening new channel: b'varia=asdf\x90 ./echo.sh': Done

In [4]: _run1.recvall()                                                                              
[x] Receiving all data
[x] Receiving all data: 0B
[x] Receiving all data: 12B
[+] Receiving all data: Done (12B)
[*] Closed SSH channel with localhost
Out[4]: b'varia=asdf\x90\n'

In [5]: _run2 = _ssh.run(b'./echo.sh',env={'varia':b'asdf\x90'})                                     
[x] Opening new channel: b'./echo.sh'
[+] Opening new channel: b'./echo.sh': Done

In [6]: _run2.recvall()                                                                              
[x] Receiving all data
[x] Receiving all data: 0B
[x] Receiving all data: 18B
[+] Receiving all data: Done (18B)
[*] Closed SSH channel with localhost
Out[6]: b"varia=b'asdf\\x90'\n"

In [7]: _run3 = _ssh.run(b'./echo.sh',env={'varia':'asdf\x90'})                                      
[x] Opening new channel: b'./echo.sh'
[+] Opening new channel: b'./echo.sh': Done

In [8]: _run3.recvall()                                                                              
[x] Receiving all data
[x] Receiving all data: 0B
[x] Receiving all data: 13B
[+] Receiving all data: Done (13B)
[*] Closed SSH channel with localhost
Out[8]: b'varia=asdf\xc2\x90\n'
```
![ipython_output_2](https://user-images.githubusercontent.com/61562700/75494948-a7600400-59bd-11ea-9cf7-fd74ccf7fab8.png)




Discussion:
I would strongly recommend using ssh.process if possible.  You can pass in
an environment that way, and everything is guaranteed to "just work"

*Zach Riggle*


On Thu, Feb 27, 2020 at 5:21 PM cameronotw <notifications@github.com> wrote:

> Hi,
>
> I tried to do write automagical solver of some wargame with use of
> pwntools. I got the problem when it comes to set environment variable for
> execution on remote host via ssh by system() method (aka run()).
>
> As to not disclose the challange for others, I prepared some examplary
> problem instead of mentioning wargame itself. If you want to know it
> exactly ask me on my loginname on Gmail Dot Com.
>
> Bear in mind that maybe I miss some python skill so sorry if You find it
> invalid. I also open to fix it and test.
>
> Here it goes. In local directory on builder@localhost there is small
> shell script named echo.sh which is very simple and look for environment
> variable.
>
> #!/bin/bash
>
> env | grep varia
>
> There we go with trying to get some bytes back.
>
> Python 3.8.1 (default, Jan 22 2020, 06:38:00)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.12.0 -- An enhanced Interactive Python. Type '?' for help.
>
> In [1]: from pwn import *
>
> In [2]: pwnlib.__version__
> Out[2]: '4.2.0dev'
>
> In [3]: _ssh = ssh(host='localhost',user='builder')
> [x] Connecting to localhost on port 22
> [+] Connecting to localhost on port 22: Done
> [*] builder@localhost:
>     Distro    Unknown
>     OS:       linux
>     Arch:     amd64
>     Version:  4.19.101
>     ASLR:     Enabled
>
> In [4]: _run1 = _ssh.run(b'varia=asdf ./echo.sh')
> [x] Opening new channel: b'varia=asdf ./echo.sh'
> [+] Opening new channel: b'varia=asdf ./echo.sh': Done
>
> In [5]: _run1.recvall()
> [x] Receiving all data
> [x] Receiving all data: 0B
> [x] Receiving all data: 11B
> [+] Receiving all data: Done (11B)
> [*] Closed SSH channel with localhost
> Out[5]: b'varia=asdf\n'
>
> In [6]: _run2 = _ssh.run(b'./echo.sh',env={'varia':b'asdf'})
> [x] Opening new channel: b'./echo.sh'
> [+] Opening new channel: b'./echo.sh': Done
>
> In [7]: _run2.recvall()
> [x] Receiving all data
> [x] Receiving all data: 0B
> [x] Receiving all data: 12B
> [+] Receiving all data: Done (12B)
> [*] Closed SSH channel with localhost
> Out[7]: b'varia=basdf\n'
>
> In [8]: _run3 = _ssh.run(b'./echo.sh',env={'varia':'asdf'})
> [x] Opening new channel: b'./echo.sh'
> [+] Opening new channel: b'./echo.sh': Done
>
> In [9]: _run3.recvall()
> [x] Receiving all data
> [x] Receiving all data: 0B
> [x] Receiving all data: 11B
> [+] Receiving all data: Done (11B)
> [*] Closed SSH channel with localhost
> Out[9]: b'varia=asdf\n'
>
>
> [image: ipython_output]
> <https://user-images.githubusercontent.com/61562700/75494947-a7600400-59bd-11ea-858a-d580244749c9.png>
>
> Here, we see that additional *b* letter was added in case of binary
> litteral.
>
> Bigger problem is when we provide the binary literal or string literal to
> it and one got's *b* letter and apostrophes, but second got 'latin1'
> encoding (same output with str(bytes,'latin1') ).
>
> Python 3.8.1 (default, Jan 22 2020, 06:38:00)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.12.0 -- An enhanced Interactive Python. Type '?' for help.
>
> In [1]: from pwn import *
>
> In [2]: _ssh = ssh(host='localhost',user='builder')
> [x] Connecting to localhost on port 22
> [+] Connecting to localhost on port 22: Done
> [*] builder@localhost:
>     Distro    Unknown
>     OS:       linux
>     Arch:     amd64
>     Version:  4.19.101
>     ASLR:     Enabled
>
> In [3]: _run1 = _ssh.run(b'varia=asdf\x90 ./echo.sh')
> [x] Opening new channel: b'varia=asdf\x90 ./echo.sh'
> [+] Opening new channel: b'varia=asdf\x90 ./echo.sh': Done
>
> In [4]: _run1.recvall()
> [x] Receiving all data
> [x] Receiving all data: 0B
> [x] Receiving all data: 12B
> [+] Receiving all data: Done (12B)
> [*] Closed SSH channel with localhost
> Out[4]: b'varia=asdf\x90\n'
>
> In [5]: _run2 = _ssh.run(b'./echo.sh',env={'varia':b'asdf\x90'})
> [x] Opening new channel: b'./echo.sh'
> [+] Opening new channel: b'./echo.sh': Done
>
> In [6]: _run2.recvall()
> [x] Receiving all data
> [x] Receiving all data: 0B
> [x] Receiving all data: 18B
> [+] Receiving all data: Done (18B)
> [*] Closed SSH channel with localhost
> Out[6]: b"varia=b'asdf\\x90'\n"
>
> In [7]: _run3 = _ssh.run(b'./echo.sh',env={'varia':'asdf\x90'})
> [x] Opening new channel: b'./echo.sh'
> [+] Opening new channel: b'./echo.sh': Done
>
> In [8]: _run3.recvall()
> [x] Receiving all data
> [x] Receiving all data: 0B
> [x] Receiving all data: 13B
> [+] Receiving all data: Done (13B)
> [*] Closed SSH channel with localhost
> Out[8]: b'varia=asdf\xc2\x90\n'
>
> [image: ipython_output_2]
> <https://user-images.githubusercontent.com/61562700/75494948-a7600400-59bd-11ea-9cf7-fd74ccf7fab8.png>
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/Gallopsled/pwntools/issues/1437?email_source=notifications&email_token=AAA3IGCHRDUNEN7OYEU5S6TRFBDIFA5CNFSM4K5FKU4KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IQ6PIIQ>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAA3IGDYETPYMJXX6JRZ3Y3RFBDIFANCNFSM4K5FKU4A>
> .
>

@Arusekk will know better how to fix this

Resolving patch:
diff --git a/pwnlib/tubes/ssh.py b/pwnlib/tubes/ssh.py
--- a/pwnlib/tubes/ssh.py
+++ b/pwnlib/tubes/ssh.py
@@ -69,26 +69,25 @@ def __init__(self, parent, process = None, tty = False, wd = None, env = None, r
         self.process = process
         self.cwd  = wd or '.'
         if isinstance(wd, six.text_type):
-            wd = wd.encode('utf-8')
+            wd = context._encode(wd)
 
         env = env or {}
         msg = 'Opening new channel: %r' % (process or 'shell')
 
         if isinstance(process, (list, tuple)):
-            process = b' '.join((lambda x:x.encode('utf-8') if isinstance(x, six.text_type) else x)(sh_string(s)) for s in process)
+            process = b' '.join(context._encode(sh_string(s)) for s in process)
         if isinstance(process, six.text_type):
-            process = process.encode('utf-8')
+            process = context._encode(process)
 
         if process and wd:
             process = b'cd ' + sh_string(wd) + b' >/dev/null 2>&1;' + process
 
         if process and env:
             for name, value in env.items():
-                if not re.match('^[a-zA-Z_][a-zA-Z0-9_]*$', name):
+                nameb = context._encode(name)
+                if not re.match(b'^[a-zA-Z_][a-zA-Z0-9_]*$', nameb):
                     self.error('run(): Invalid environment key %r' % name)
-                export = 'export %s=%s;' % (name, sh_string(value))
-                if isinstance(export, six.text_type):
-                    export = export.encode('utf-8')
+                export = b'export %s=%s;' % (nameb, sh_string(context._encode(value)))
                 process = export + process
 
         if process and tty:
@@ -264,11 +263,11 @@ def recv_thread(event):
                 if not data:
                     event.set()
                 else:
-                    data = [six.byte2int(data)]
+                    data = bytearray(data)
 
             if data:
                 try:
-                    self.send(b''.join(six.int2byte(c) for c in data))
+                    self.send(bytes(bytearray(data)))
                 except EOFError:
                     event.set()
                     self.info('Got EOF while sending in interactive')
@@ -1154,6 +1153,8 @@ def system(self, process, tty = True, wd = None, env = None, timeout = None, raw
             >>> py.sendline(b'exit')
             >>> print(repr(py.recvline()))
             b'4\n'
+            >>> s.system('env | grep -a AAAA', env={'AAAA': b'\x90'}).recvall()
+            b'AAAA=\x90\n'
         """
 
         if wd is None:


Test patch: