Py3 failing tests

Fixes the failing tests revealed by #4165. This sits on top of that PR, though, so the diff won't be useful until that lands.