Bug[1215560] Remove pip package from pip cache

Do not merge until Bug[1215560] pull 9 has merged