Maybe we should make this into a real error, not an assert. What do you think?