[WIP] remove check_arrays stuff and old input validation

Follow-up on #3443, removing all the old stuff.
You can remove the order here