Add assert about the shape in the graph.

fix gh-1738
Fix and add an assert
Add tests that we don't remove errors for all supported ndim.
Merge pull request #1740 from nouiz/shape_err

Add assert about the shape in the graph.