Remove matplotlib from pip requirements

The RTD build failed on this install and we are not currently using it.