Remove matplotlib from build_versions