Repository: Gallopsled/pwntools

Issue:
sendfile doesn't seems to be able to read /proc/self/maps in recent kernel version
Hi, after some experiment it seems `sendfile` system call can not read `/proc/self/maps` anymore in recent kernel version (tested in archlinux x86 5.11, archlinux arm 5.10). Here is a simple c code to test:
```c
#include <stdlib.h>
#include <unistd.h>
#include <sys/sendfile.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>

int main(int argc, char *argv[])
{
	if (argc != 2) {
		printf("Usage: %s <file>\n", argv[0]);
		exit(1);
	}
	int fd = open(argv[1], O_RDONLY);
	int out_fd = creat("/dev/stdout", S_IRUSR|S_IWUSR);  
	if (fd < 0) {
		perror("open file");
		exit(1);
	}
	if (out_fd < 0) {
		perror("open stdin");
		exit(1);
	}
	int ret = sendfile(out_fd, fd, NULL, 1024);
	if (ret == -1) {
		perror("sendfile");
		exit(1);
	}
	
	return 0;
}

```
It prints "invalid argument" in 5.10/5.11 but works fine in a 4.15 system.
I could not find the source of this change, so I'm not sure if this is some kind of upstream bug or is intended. However this renders shellcraft amd64 cat stops working, which has an influence on ELF libs/libc (alwasy empty or `None`).


Discussion:
Apparently, I just tested that amd64 has the exact same issue.
So maybe its related to this commit: https://github.com/torvalds/linux/commit/36e2c7421f02
According to some discussion in LKML [mails], splice support for some of the proc files were added, but not including `/proc/pid/maps`. However even if this is supported, supplying stdout fd (1) as `out_fd`  directly will also result in an `invalid argument`.

[mails]: https://lore.kernel.org/lkml/20201027074911.GB29565@infradead.org/
It may be related to stdout being a terminal. Does it work if you invoke the program with an explicit pipe or socket at stdout?
Like `./a.out | cat` or `socat - exec:./a.out` (don't use `creat("/dev/stdout")` for it, use `dup(STDOUT_FILENO)`).
Yep, that seems to be the issue. I tested in more systems and only one virtualbox vm needs to piping to cat or using socat to make it works. Others I can just invoking on the command line directly without issue. Not sure why though.
So now the problem remains is that sendfile can't read /proc/self/maps...
This looks bad, since neither `copy_file_range(2)` nor `splice(2)` look like an option to us now, and the shellcode does not want to allocate super-much memory, especially if it resides on stack; we would need to agree on some buffer size, I think 4096 (single page) would be best, but should it be on stack? Or mmapped? Or should we `mmap` the input file instead of `read`ing it?
I think the `CAT_PROC_MAPS_EXIT` pre-existing shellcodes can be replaced with mmap-then-write (or a read-write loop if they also broke `seq_ops` mmap, not tested), but I would leave `shellcraft.linux.cat` using `sendfile` and provide a separate `slowcat`/`copycat`/`reliable_cat` that would allocate a stack page (or mmap some anon memory) and do a read-write loop. Or maybe better `cat`, `mmapcat` and `readcat` doing the respective. It would be terrible to have read-write for `CAT_PROC_MAPS_EXIT`, though, since pwntools' ELF handling is pretty slow already and any low-hanging speedups are invaluable.

Note that I especially don't like the very fact this was broken in the kernel. I hope someone submits a patch adding it back (looks like some [trivial](https://lore.kernel.org/lkml/20201028183441.GA8461@infradead.org/) [modifications](https://lore.kernel.org/lkml/20201028183525.GB2957431@kroah.com/)), maybe even I will if I manage to find some spare free time.
So I think this is a kernel bug, and we probably can work it around, but should be treated as a kernel bug. Here is our punishment for not testing pwntools on kernel release candidates.

Is it exposed on some test in our test suite? Do they run fine in your containers?
I agree with you. It would be the best if they could add it back in kernel, and it seems reasonable given some other proc files' support are added back already.
As for the tests I will take a run in my environment later and report the result. 

I need to add a `RUN sudo apt-get update` in dockerfile in order to make the image successfully built. After running test in docker there're 15 tests failed, 14 of them are in `elf/corefile` which I believe is because of missing command `coredumpctl`. The last failed case is indeed revealing the problem in this issue (`bash.libs` is empty, and a manual repl run behaves the same):
```
File "../../pwnlib/elf/elf.py", line ?, in default
Failed example:
    any(map(lambda x: 'libc' in x, bash.libs.keys()))
Expected:
    True
Got:
    False
/usr/bin/x86_64-linux-gnu-objcopy: /tmp/pwn-asm-7vmw302o/sty1GluE: section .A lma 0x8049074 adjusted to 0x804a054

```
My first submitted kernel patch; hope they like it. 😅 You can watch the [lkml thread](https://lore.kernel.org/lkml/20210427183414.12499-1-arek_koz@o2.pl/) if you want; probably few will notice the patch through the storm of `@umn.edu` reverts.
> You can watch the lkml thread 

will do :) This looks promising
Thanks @Arusekk for submitting the patch to the kernel!  Hopefully it gets integrated quickly, else I suspect we'll see lots of issue requests for this.

It would be a LOT of work, but we can change the `CAT_PROC_MAPS_EXIT` shellcode to work correctly in all cases, since there are no shellcode size restrictions.  

This would require us writing a special-case that does a stack reservation, `read(2)`, and `write(2)`, for each individual architecture, and maintaining it.  

I believe that the current code is just a hard-coded copy of e.g. `shellcraft.arm.linux.cat("/proc/self/maps")` so there's no maintenance burden.  Hard-coding was done for (1) speed [don't need to assemble at exploit-runtime] and (2) avoiding the need for the user to have the correct assembler to use it.  

We could call the resulting shellcraft template `catproc.asm`.  I would support backporting such a change to `stable`.

If we do, it *might* be worth adding a specific check for "/proc" to each `cat.asm` and have it emit a warning message that the generated shellcode may not work on kernel version XYZ, and suggest they use `shellcraft.catproc` instead.
It appears the proposal to add `sendfile` support back to `/proc/pid/maps` won't be accepted, at least in the near future :(
So for now maybe the only thing we can do is to change the shellcode in order to deal with this?
I talked with @Arusekk a bit, and I think the approach we'll adopt will be to rewrite the CAT_PROC_PID_MAPS shellcode to use `read` and `write` using the stack as a buffer.

One way we discussed to achieve this is to modify `cat.asm` for each supported architecture to have a `mode=` operation, which uses `sendfile` by default but if `mode='compatible'` (or some other such flag) is specified, doing `open / read / write` will be used instead.

This requires writing new shell code for all of the architectures, and might take a while since it's a significant time investment.
Additional restrictions on `LD_TRACE_LOADED_OBJECTS` is that it doesn't work for BSD, musl linker, or Android crazy_linker.

A workaround is to use `gdb.find_module_addresses`, and load the ELFs yourself.  However, this does not work for cross-arch-emulated-with-qemu binaries because it launches GDB directly rather than using `gdb.debug()` with a gdbscript.  It also requires you have a GDB capable of debugging the binary (e.g. vanilla GDB might not support mips64).

Resolving patch:
diff --git a/pwnlib/elf/elf.py b/pwnlib/elf/elf.py
--- a/pwnlib/elf/elf.py
+++ b/pwnlib/elf/elf.py
@@ -767,7 +767,7 @@ def _patch_elf_and_read_maps(self):
             >>> for arch in CAT_PROC_MAPS_EXIT:
             ...   context.clear()
             ...   with context.local(arch=arch):
-            ...     sc = shellcraft.cat("/proc/self/maps")
+            ...     sc = shellcraft.cat2("/proc/self/maps")
             ...     sc += shellcraft.exit()
             ...     sc = asm(sc)
             ...     sc = enhex(sc)
diff --git a/pwnlib/elf/maps.py b/pwnlib/elf/maps.py
--- a/pwnlib/elf/maps.py
+++ b/pwnlib/elf/maps.py
@@ -3,25 +3,25 @@
 # Pre-assembled shellcode for each architecture.
 #
 # This is literally the output of:
-#     shellcraft $ARCH.linux.cat /proc/self/maps
+#     shellcraft $ARCH.linux.cat2 /proc/self/maps
 #     shellcraft $ARCH.linux.syscalls.exit 0
 CAT_PROC_MAPS_EXIT = {
     'i386':
-        '680101010181342460717201686c662f6d68632f7365682f70726f89e331c931d26a0558cd806a015b89c131d268ffffff7f5e31c0b0bbcd80'
+        '680101010181342460717201686c662f6d68632f7365682f70726f89e331c931d2b6406a0558cd8029d489c389e16a0358cd806a015b89e189c26a0458cd80'
         '31db6a0158cd80',
     'amd64':
-        '48b801010101010101015048b86d672e6c607172014831042448b82f70726f632f7365506a02584889e731f6990f0541baffffff7f4889c66a28586a015f990f05'
+        '48b801010101010101015048b86d672e6c607172014831042448b82f70726f632f7365506a02584889e731d2b64031f60f054829d44889c731c04889e60f054889c26a01586a015f4889e60f05'
         '31ff6a3c580f05',
     'arm':
-        '617007e3737040e304702de56c7606e32f7d46e304702de5637f02e3737546e304702de52f7007e3727f46e304702de50d00a0e1011021e0022022e00570a0e3000000ef0010a0e10100a0e3022022e00231e0e3bb70a0e3000000ef'
+        '617007e3737040e304702de56c7606e32f7d46e304702de5637f02e3737546e304702de52f7007e3727f46e304702de50d00a0e1011021e00129a0e30570a0e3000000ef02d04de00d10a0e10370a0e3000000ef0020a0e10100a0e30d10a0e10470a0e3000000ef'
         '000020e00170a0e3000000ef',
     'thumb':
-        '004f01e0617073ff4fea07274fea172780b4dff8047001e06c662f6d80b4dff8047001e0632f736580b4dff8047001e02f70726f80b4684681ea010182ea02024ff0050741df05464ff00100294682ea02026ff000434ff0bb0741df'
+        '004f01e0617073ff4fea07274fea172780b4dff8047001e06c662f6d80b4dff8047001e0632f736580b4dff8047001e02f70726f80b4684681ea01014ff480424ff0050741dfadeb020d69464ff0030741df02464ff0010069464ff0040741df'
         '80ea00004ff0010741df00bf',
     'mips':
-        '726f093c2f702935f0ffa9af7365093c632f2935f4ffa9af2f6d093c6c662935f8ffa9af8cff193c9e8f393727482003fcffa9aff0ffbd272020a003ffff0528ffff0628a50f02340c010101feff192427202003fcffa2affcffa58fffff0628ff7f073cffffe7346f1002340c010101'
+        '726f093c2f702935f0ffa9af7365093c632f2935f4ffa9af2f6d093c6c662935f8ffa9af8cff193c9e8f393727482003fcffa9aff0ffbd272020a003ffff0528ffbf192427302003a50f02340c01010122e8a603fcffa2affcffa48f2028a003a30f02340c010101feff1924272020032028a003fcffa2affcffa68fa40f02340c010101'
         'ffff0428a10f02340c010101',
     'aarch64':
-        'ee058ed24eeeadf26eecc5f26eaeecf28fcd8cd2efa5adf22f0ccef26f0ee0f2ee3fbfa980f39fd2e0ffbff2e0ffdff2e0fffff2e1030091e2031faae3031faa080780d2010000d4e10300aa200080d2e2031faae3ff9fd2e3ffaff2e80880d2010000d4'
+        'ee058ed24eeeadf26eecc5f26eaeecf28fcd8cd2efa5adf22f0ccef26f0ee0f2ee3fbfa980f39fd2e0ffbff2e0ffdff2e0fffff2e1030091e2031faa080780d2010000d4020088d2ff6322cbe1030091e80780d2010000d4e20300aa200080d2e1030091080880d2010000d4'
         'e0031faaa80b80d2010000d4',
 }


Test patch: