Remove spec test directory from gem.