fix `setup.py submodule`

closes #3257