Phylogenetic penalized likelihood