Remove some asserts which don't make sense anymore.