Unrelated to this PR, but we shouldn't be using assert here. Either we should nix these two lines or change them to raise non AssertionError exceptions in the error cases.