  Standard likelihood penalties to learn Gaussian graphical models are based on
regularising the off-diagonal entries of the precision matrix. Such methods,
and their Bayesian counterparts, are not invariant to scalar multiplication of
the variables, unless one standardises the observed data to unit sample
variances. We show that such standardisation can have a strong effect on
inference and introduce a new family of penalties based on partial
correlations. We show that the latter, as well as the maximum likelihood, $L_0$
and logarithmic penalties are scale invariant. We illustrate the use of one
such penalty, the partial correlation graphical LASSO, which sets an $L_{1}$
penalty on partial correlations. The associated optimization problem is no
longer convex, but is conditionally convex. We show via simulated examples and
in two real datasets that, besides being scale invariant, there can be
important gains in terms of inference.
