Remove unnecessary assertions

These are not important but sometimes fail