Repository: google/oss-fuzz

Issue #7677: ujson fuzzing probably ineffective due to implicit debug symbol stripping
Created: 2022-05-06T11:34:28Z
IIUC https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=36009 correctly, the relevant report has been closed as resolved due to ultrajson/ultrajson@e3ccc5a...682c660.
However, this commit range doesn't seem to contain any valid fix — it merely starts stripping debugging symbols on Linux. Therefore, I suspect that this implicit stripping has made the fuzzer non-operational.

Duplicate issue #7723: Python bugs raised by oss-fuzz are currently unhelpful for OSS maintainers.
Created: 2022-05-19T21:12:35Z
Hi, I was recently involved in fixing a CVE detected by oss-fuzz (namely this one) on a package I co-maintain and to be honest, oss-fuzz did not help us at all. However, I really don't think it would take much to change oss-fuzz reports from near useless to priceless.
The biggest pain point was the reproducer and figuring out what the bug it was reporting was. Ideally, I'd expect a report like ujson.dumps(), if fed parameters x, y and z, segfaults. If that was written in the report then I'd singing praises to oss-fuzz. Instead we got this strange binary testcase file. There's no guidance in the report about what that file is or how to interpret it - after about half a day of digging I found the docs (here) hiding away under advanced topics - having that link in the report would have been a big help. That tutorial is pretty good; I was able to get oss-fuzz to run locally, run the test case and re-detect the segfault. Unfortunately the exercise turned out to be all in vain since I still gain any insight on what it was doing to trigger buffer overflow. Yesterday someone suggested to me a hack to get that information out. If there's no better way of doing it than that then PLEASE add one. If there is, please make it more visible in the documentation. In the end, I threw in the towel and abandoned the fuzz report and test case. Another maintainer had heard vague reports from someone else that there were some memory issues surrounding ujson.dumps() with very large indentations. Based on that, I reinvented the wheel and created my own ad-hoc fuzzer which finally gave us the reproducer that we needed to fix the issue.
A second, more minor pain point was the introduced and fixed commit SHAs. In all honesty, these wouldn't have been that important to me if I weren't hoping that they might give me a clue as to what the issue really was. The breaking commit doesn't even touch the source code and the fixing commit is in a completely unrelated project. Urmm 😕?
I hope that this is constructive rather than just whiny feedback. oss-fuzz appears to have so much potential but, given that every key piece of information in that report was wrong, all it did was confuse us maintainers and lead to conversations with some very panicked users along the lines of [User]: Is CVE-2021-45958 fixed yet? What's the fix version? [Me]: I have no idea. I still don't even know what CVE-2021-45958 is. 🙃

Closing comment (2022-05-23T20:32:12Z): Regarding reproduction, we provide this text in all of our bug reports ("See https://google.github.io/oss-fuzz/advanced-topics/reproducing for instructions to reproduce this bug locally.") in all bug reports, but it probably needs to be made prominent.

Grr, I knew there would be a facepalm moment like this. I guess that it would help a bit just to move that sentence next to the link to the reproducer. I don't suppose those reports support any kind of basic markup? If you are allowed then something like Reproducer Testcase : 5751832088543232 (see the reproducer guide for what to do with this file) should be pretty fool proof.
On the subject of links, there's the one at the top (Detailed Report: https://oss-fuzz.com/testcase?key=5751832088543232). It takes me through a google sign in then tells me that I don't have access to proceed. What detailed report is on the other side of that link? Are the package authors supposed to be able to see it? It's a bit of a road to nowhere having it there if access is limited.

It seems like it's difficult to understand the inputs passed to this target, as the inputs are in an opaque data format created by hypothesis.

Ahh, it's hypothesis producing that file. The reasons why things are the way they are are becoming clearer. I had assumed that this was a custom oss-fuzz specific file and was going to suggest that if you swapped it for something text based and reasonably universal like yaml then package authors could probably get the gist of what the offending test case is just by looking at it - that's the end of that line of thought.

While our automation infra works for most cases, it does hiccup occassionally. This may be related to #7677, which made it difficult for our infra to bisect properly.

In ujson's case, there's also the fact that memory bugs could move around without being fixed. The security hole in ujson.dump() arised from ujson underestimating how much more buffer space it would need but, because the buffer was only expanded by doubling its size, a segfault would only occur if the range from estimated required buffer size to actually required buffer size straddled an integer power of 2. Slight changes in ujson's formatting over time shifts that range for any given reproducer onto or off a power of 2 so a bisection running only the reproducing test case would be doomed.