fix build

The build currently fails until I separate numpy to install 1st.
I see something like this has already happened and solved the same with pandas/statsmodels.
The --upgrade suppresses one of the warnings, not sure if it works without or not, I think it fails due to version dependency. 
The clean here is to actually remove the files from the FS layer.
Each commands creates a new layer.
If you have it in the end you are both adding it and subtracting it in a different layer.
That's some docker craziness.
You could squash it in the end for better performance and less storage I head...