[WIP] remove check_arrays stuff and old input validation

Follow-up on #3443, removing all the old stuff.
I got trapped by the current weird behavior.