Gaussian Differentially Private Human Faces Under a Face Radial Curve Representation

## Abstract

Abstract In this paper we consider the problem of releasing a Gaussian Differentially Private (GDP) 3D human face. The human face is a complex structure with many features and inherently tied to one’s identity. Protecting this data, in a formally private way, is important yet challenging given the dimensionality of the problem. We extend approximate DP techniques for functional data to the GDP framework. We further propose a novel representation, face radial curves, of a 3D face as a set of functions and then utilize our proposed GDP functional data mechanism.
To preserve the shape of the face while injecting noise we rely on tools from shape analysis for our novel representation of the face.
We show that our method preserves the shape of the average face and injects less noise than traditional methods for the same privacy budget. Our mechanism consists of two primary components, the first is generally applicable to function value summaries (as are commonly found in nonparametric statistics or functional data analysis) while the second is general to disk-like surfaces and hence more applicable than just to human faces.

## 1 Introduction

In statistical analyses, data and parameters appear in varying degrees of complexity, from simpler forms such as scalars and vectors to more complex such as spherical or hyperbolic, for instance. The structural constraints inherent to data need to be respected throughout any analysis as has been shown in the “intrinsic statistics" frameworks (Pennec, ; Bhattacharya and Patrangenaru, ) for accurate estimation and to preserve said structure.
Complex data structures tend to come hand in hand with complex statistical computations and hence the techniques for handling such data have not been widely studied outside of specific scenarios.
Further, the sheer amount of data that is captured from individuals has increased significantly, and of course, this produces a growing concern for one’s “privacy". In this paper, to support broader sharing of confidential data, we propose releasing a Gaussian Differentially Private, GDP, average 3D human face.

Motivation and Related Literature:
Data that live in nonlinear spaces can be challenging to work within the DP framework, as shown in the context of manifolds in Reimherr et al. (); Soto et al. (); Utpala et al. (), private Riemannian optimization in Han et al. (), and Gaussian DP on manifolds Jiang et al. (). Preservation of structure has also been considered in the context of private covariance estimation for linear regression in Sheffet () and private principal component analysis in Chaudhuri et al. () which is connected to the Stiefel manifold. For privacy, our proposed “face radial curve" representation of a human face are functions extracted from a disk and hence lends itself to be examined under the lens of private functional data analysis (FDA) which has been considered inWasserman and Zhou () and Mirshani et al. (), and references therein.

With our faces constantly being captured (e.g., at a grocery store self-checkout), one could question the privacy protections in place of the collected data. Further, there is a vast amount of literature on identification of individuals from facial data in the area of “face recognition," see, for instance, the expansive literature review in Kortli et al. (). Also, perhaps quite surprisingly, Venkatesaramani et al. () and Klimentidis and Shriver () showed that one could identify genomic information from a person’s 2D face image; the former study further showed that adding noise to said 2D images can help protect this re-identification. We do not work with 2D images here, however, the need for privacy is still present. The need for privacy for 3D faces is similar to that in 2D images. Anthropological studies are often interested in identification of DNA using labeled 3D faces (Sero et al., ), the connections between DNA genotype and the associated facial phenotype (White et al., ; Naqvi et al., ; Weinberg et al., ), and average faces for demographics such as age and race classification (Tokola et al., ). In such studies, one might want to release the data, or may even be required to, so offering provable confidentiality guarantees for people in the studies becomes an important task.

To generate our representation we rely on tools from shape analysis. The earlier forms of shape analysis, such as Kendall’s shape space Kendall (), are limited to only representing a shape as a finite point cloud. The field has expanded since to consider more complex data structures such as continuous curves, both planar and space, as in Trouvé and Younes (); Klassen et al. (); Srivastava et al. (), and surfaces as in Jermyn et al. (); Su et al. (). Human faces have been considered in this space in a similar, yet subtly different, manner such as in Samir et al. (); Drira et al. () in which faces are represented as a set of curves “facial curves" and “radial curves", respectively. These methods represent a face with curves that are generated independently of each other while we use an entire disk-parameterization to capture features across all faces simultaneously.

Main Contributions:
We develop a novel representation for a collection of 3D faces via a set of curves which we extract from disk parameterizations and refer to these as face radial curves. We construct the face radial curves using tools from statistical shape analysis in the interest of preserving the shape of an average face during the data sanitization process.
Further, we extend existing approximate DP FDA tools into the Gaussian DP framework (Dong et al., ), a recent notion of privacy with attractive properties including “tight" composition. Under our FDA Gaussian DP mechanism, we generate a private average face of a collection of faces under our representation. While we use our mechanism and representation for faces to address the inherent data privacy concerns, this same methodology can be applied to any surfaces diffeomorphic to a unit disk such as terrain models and additionally any settings where one wants private functional statistical summaries.

## 2 Notation and Background on Face Representation

We require a disk-parameterized representation of each face from which we extract a set of functions. We parameterize each face independently but “align" and “register" each face to a template. Here, we broadly describe these relevant techniques from shape analysis including their necessity.
To the best of our knowledge, the use of parameterized surfaces in the context of DP has not been explored. The software to accomplish parameterization, registration, and alignment are fully described in Jermyn et al. (), with accompanying implementation at GitHub repository (Laga, ).

### 2.1 Parameterization

Each face is realized as a point cloud, a set of $p$ many points in $\mathbb{R}^{3}$. We describe the data collection process in A.4. A point cloud does not explicitly relay structural information; i.e., there is no natural ordering nor explicit connectivity or relationship between points. Connectivity can be difficult to infer since if two points are close in space, they may not be neighboring points, e.g., two points near the tip of the nose, or any concave feature, can be close in $\mathbb{R}^{3}$, but based on measuring distance on the surface of the face they may be relatively far apart. So, rather than treat a face as a set of independent points, we can jointly model each face by imposing a parameterization. Specifically, we represent each face as a disk-parameterized surface (Jermyn et al., ; Sun et al., ; Drira et al., ).

Figure: Figure 1: Left: A triangulated mesh face. Middle: A disk-parameterized face, the center of the disk being the left nostril. Right: A disk-parameterized face after a Möbius transformation forcing the center of the disk to be the tip of the nose.
Refer to caption: x1.png

We first compute a Delaunay triangulation using the MeshLab software (Cignoni et al., ) yielding a triangulated mesh. Given a point cloud $X\in\mathbb{R}^{p\times 3}$ the triangulated mesh, $M=\{V,E,T\}$, is an object which consists of a set of vertices $V$ (the original points), edges $E$, and triangles $T$ composed of said vertices and edges. The triangles meet at edges, do not overlap, and jointly represent a mesh or surface. The left panel of Figure 1 displays a face as a triangulated mesh. We generate a disk conformal map of $M$ as in Choi and Lui (); a disk conformal map of the triangulated mesh is a mapping from said mesh to a disk which preserves the angles of the triangles or, more generally, the local geometry of the mesh.

Lastly, we generate a disk-parameterized surface as in Jermyn et al. (); Laga et al. () via the disk conformal mapping. Figure 1 displays two disk-parameterized surfaces in the middle and right panels; we expand on the differences between these two panels in A.6.
Each face, $f$, is thus a map $f:D\rightarrow\mathbb{R}^{3}$ with $D=\{r,\theta|0\leq r\leq 1,0\leq\theta<2\pi\}$, the unit disk.

### 2.2 Shape Analysis of Surfaces

Figure 1 displays different representations of a face while retaining its shape. In our setting, the shape of an object is that which is not affected by its scale, location, rotation, or parameterization. The analysis we intend to do should not be dependent on any of these parameters. Next, we describe how to remove this dependence.

Let $\mathcal{F}$ be the space of all parameterized surfaces, $\mathcal{F}\ni f:D\rightarrow\mathbb{R}^{3}$ where $D$ is the unit disk. We assume all surfaces are smooth and genus-0, that is, they are differentiable and have no holes. We first remove scale and location differences by forcing each face to have unit surface area and be centered at the origin. That is, we scale the surface area to be one by setting $f\rightarrow f/\int_{D}|f_{r}\times f_{\theta}|_{2}drd\theta$ where $f_{r},f_{\theta}$ are the partial derivatives of $f$ with respect to $r$ and $\theta$, respectively. To center the surface, set $f\rightarrow f-\bar{f}$ where $\bar{f}$ is the centroid of the surface. For notational simplicity, let $f$ denote a surface which has unit surface area and the origin as its centroid.

Location and scale are characteristics that are intrinsic to each surface, so we achieve their removal on each face independently. Rotational alignment and parametric registration, however, are relative. Each face must be aligned and registered to a template face.(^1^11In shape analysis it is more typical to do pairwise alignment and registration (Wallace et al., ; Cho et al., ; Klassen et al., ) however our goal is not to do pairwise comparisons so we forego this approach.) Let $f_{temp}$ be a template face of unit area and origin centroid; in practice one could either choose an arbitrary surface from the dataset, a training set, or some representative surface such as the mean surface.

Let $\mathbb{O}=\{O|\text{det}(O)=1\}$ be the set of all $3\times 3$ rotation matrices and let $\Gamma=\{\gamma|\gamma:D\rightarrow D\}$ be the set of diffeomorphisms. Each $\gamma$ is a reparameterization of a surface which acts on a surface on the right as $f\circ\gamma$; we fully describe this action in A.3. Rotations, $\mathbb{O}$, act on the left as $O\cdot f$ and do not effect scale nor centroid.

We optimally align each face to the template, $f_{temp}$, by considering the optimization

$$ $(\tilde{\gamma},\tilde{O})=\operatorname{argmin}_{\gamma\in\Gamma,O\in\mathbb{ O}}\|Of\circ\gamma-f_{temp}\|_{2}.$ $$

To accomplish this task we appeal to elastic shape analysis (ESA) (Jermyn et al., ). In short, rather than solve the above minimization, ESA defines a metric which replaces the above $\mathbb{L}^{2}$ norm and transforms each surface using the square-root normal field representation. For each face we compute this minimization such that the registered and aligned face is $\tilde{f}:=\tilde{O}f\circ\tilde{\gamma}$. We elaborate on this process in A.3.

## 3 Novel Face Radial Curves Representation

We parameterize the entire face using a disk, however, our goal is to work with what we refer to as “face radial curves." This concept is similar to that of facial curves (Samir et al., ) and radial curves (Drira et al., ). In the facial recognition literature, for instance, to construct a radial curve representation one picks a focal point, typically the tip of the nose, and overlays curves on the surface of the face with the nose as the center and each point of the curve being equidistant from the tip of the nose. That is, given a focal point $p$ each radial curve is $g=\{x|r=d_{F}(p,x)\}$ for a given $r\geq 0$ and where $d_{F}$ is distance measured on the surface of the face.

These previously mentioned methodologies do not take into account that the distribution of features on faces is not uniform for everyone. That is, let $g_{ij}$ represent the $j$th radial curve of the $i$th face, then $g_{ij}$ may be the curve which goes directly on the middle of the eyes for individual $i$ but $g_{i^{\prime}j}$ may be directly on the forehead of individual $i^{\prime}$ for the same $j$. Features, such as eyes and the mouth, being disproportionately farther or closer to the focal point or each other hence causes issues. These methodologies do, however consider registration within each $j$. Therefore, registration has been considered within each set of curves $j$ but not across all $j$s simultaneously. We propose a new method to have an entire global alignment and registration across all sets of curves and faces.

In the previous section, we set up a way to compute faces $\{\tilde{f}^{(i)}\}_{i=1,\cdots n}$ that are registered and aligned to a template face $f_{temp}$.
Each face is a mapping $\tilde{f}^{(i)}:D\rightarrow\mathbb{R}^{3}$ and we note a disk is an infinite collection of concentric circles, and thus each $\tilde{f}^{(i)}$ is a collection of curves $\tilde{f}^{(i)}_{r}:D_{r}\rightarrow\mathbb{R}^{3}$, where $D_{r}\subset D$ is the circle of radius $r$. By construction, the alignment and registration to a template implies these curves $\tilde{f}^{(i)}_{r}$ capture the same features across all faces; these curves are what we refer to face radial curves.

We apply our proposed method to data described in Sero et al. (); more details are available A.4. The left panel of Figure 2 displays a face as a point cloud and the right panel is the same face but with the disk parameterization and some of the face radial curves overlain. While the disk parameterization is an infinite collection of these curves, we pick some number of these curves to represent the face. The number of curves is a tuning parameter where more curves means more definition in the face but only to an extent. We discuss this further in 5.

Figure: Figure 2: Left: A point cloud representation of a face. Right: A surface representation of the face with an overlay of radial curves.
Refer to caption: x4.png

## 4 Gaussian Differential Privacy for Functional Data

Since the conception of DP (Dwork et al., ), noise calibration has been considered from different perspectives leading to many variants.
For instance, “zero concentrated differential privacy" (Bun and Steinke, ) and “Rényi differential privacy" (Mironov, ) are notions of DP from the perspective of a divergence of the distribution of the mechanism. In the present paper we utilized “Gaussian DP" (GDP, Dong et al. ()), which considers DP from the perspective of a particular set of hypothesis tests. As one of our key contributions in this paper, we extend GDP to functional data analysis (FDA) in Section 4.2. We begin with a brief overview of DP for FDA.

### 4.1 Background on Differential Privacy for Functional Data

In this section and in A.5, we describe approximate DP, $(\epsilon,\delta)$-DP, in the context of FDA. This is a needed background for our proposed extension of GDP into the space of functional data. For a more thorough exposition on DP, see Blum et al. (); Dwork and Roth (); Dwork et al. ().

Let $D$ denote a dataset of a size $n$, $D=\{x_{1},x_{2},\dots,x_{n}\}$, and $\mathcal{D}$ be the space of all such datasets. An adjacent dataset, $D^{\prime}$, is a dataset which differs from $D$ in exactly one element which, without loss of generality, we can choose as the last element, $D^{\prime}=\{x_{1},x_{2},\dots,x_{n}^{\prime}\}$. We write $D\sim D^{\prime}$ to denote adjacency.
Here $h(D)$ denotes the statistical summary we aim to release, and a private random version of the summary as $\tilde{h}(D)$, which we will refer to as a privacy mechanism.

We consider releasing an estimate of a private mean function. To sanitize functions we rely on tools and foundations of functional data analysis in the domain of privacy as in Hall et al. (); Alda and Rubinstein (); Mirshani et al. (), with the latter considering spaces more extensive than functions. The infinite dimensional nature of function valued summaries presents a critical challenge in establishing formal privacy. In particular, traditional probability densities become much more complicated as there is no default or baseline measure in infinite dimensions (unlike Lebesgue measure in $\mathbb{R}^{d}$). To overcome this challenge, there are currently two approaches. The first, taken by Hall et al. () is to work in finite dimensions and then take careful limits. The second approach, introduced in Mirshani et al. () and which we follow here, is to utilize carefully constructed infinite dimensional densities so that the probability inequalities can be worked out directly (and avoid having to take limits).

Let $\mathbb{H}$ denote a real separable Hilbert space in which we aim to release a summary statistic $h(D)\in\mathbb{H}$, e.g., $\mathbb{L}^{2}([0,1])$, $\mathbb{R}^{n}$, or a reproducing kernel Hilbert space. We consider a Gaussian process in $\mathbb{H}$ to add noise to $h(D)$.
Let $X$ be a Gaussian process in $\mathbb{H}$ parameterized by its mean $\eta\in\mathbb{H}$ and covariance operator $C:\mathbb{H}^{*}\rightarrow\mathbb{H}$ where $\mathbb{H}^{*}$ is the dual space of $\mathbb{H}$. A stochastic process is said to be Gaussian if any linear functional $a\in\mathbb{H}$ of $X$ is Gaussian in $\mathbb{R}$. We note that technically $a\in\mathbb{H}^{*}$, however since $\mathbb{H}\cong\mathbb{H}^{*}$, we avoid this distinction unless necessary. Further, we have that $\mathbb{E}[a(X)]=a(\eta)$ and $C(a,b)=\text{Cov}(a(X),b(X))$ with $a,b\in\mathbb{H}$. We write that $X\sim\mathcal{N}(\eta,C)$ and $Z\sim\mathcal{N}(0,C)$. In this setting we can achieve approximate DP via Theorem A.2. A critical requirement is that our summary be compatible with the noise $Z$ which, roughly stated, means that while the noise lives in $\mathbb{H}$, we require our summary to exists in a smaller space $\mathcal{H}\subset\mathbb{H}$. In our case $\mathcal{H}$ is a reproducing kernel Hilbert space (RKHS) so the eigenfunctions arise from $C$ and thus are determined by our choice of kernel $k(s,t)$.

Now that we have well defined noise, we consider realeasing a specific summary statistic: a private mean function with sample mean $h(D)=\bar{X}=\frac{1}{n}\sum_{i}x_{i}$. To have some control on the smoothness of this estimate, we enforce smoothness by relying on an optimization problem with a penalty. That is, we let $h(D)=\operatorname{argmin}_{m\in\mathcal{H}}\sum_{i}\|x_{i}-m\|^{2}_{\mathbb{H
}}+\phi\|m\|_{\mathcal{H}}^{2}$ with $\phi$ being a smoothness penalty parameter. This approach ensures the required noise compatibility.
For our summary statistic, the sensitivity, $\sup_{D\sim D^{\prime}}\|h(D)-h(D^{\prime})\|^{2},$ can be shown to be bounded as $\Delta^{2}\leq 4\tau^{2}/(n^{2}\phi)$ (Mirshani et al., ), where $\tau$ is a finite bound on the norm in the $\mathbb{H}$ space of the elements of all $D\in\mathcal{D}$.

### 4.2 Extension of GDP to Functional Data

One especially useful interpretation of approximate DP comes from Wasserman and Zhou (), which relates DP to hypothesis testing. In particular, for a sanitized output, $\tilde{h}(D)$, one can consider statistical tests for determining if the true underlying data source was $D$ or some adjacent data set $D^{\prime}$. In these simple cases, the optimal test is well known (Neyman-Pearson Lemma), so one can talk about the optimal type 2 error (1-power) of a statistical test for given type 1 error rate. It turns out that DP gives a bound for the type 2 error relative to the a type 1 error, which Dong et al. () took a step further to define Gaussian DP. In particular, Dong et al. () show a mechanism is $\mu$-GDP if its entire type 1/type 2 error tradeoff curve is bounded by that of a curve coming from comparing a $N(0,1)$ to $N(\mu,0)$, meaning it is harder to distinguish 0 0 and $\mu$ from $N(0,1)$ than it is $h(D)$ from $h(D^{\prime})$ from $\tilde{h}(D)$. We provide the formal definition below.

###### Definition 4.1 .

A mechanism $\tilde{h}(D)$ is said to satisfy $\mu-$Gaussian differential privacy ($\mu-$GDP) (Dong et al., ) if for all adjacent datasets $D\sim D^{\prime}$,

$$ $T(\tilde{h}(x;D),\tilde{h}(x;D^{\prime}))\geq G_{\mu},$ $$

where $T$ is a trade-off function and $G_{\mu}:=T(N(0,1),N(\mu,1))$. Here a trade-off function $T:[0,1]\rightarrow[0,1]$ for two probability distributions $U_{1}$ and $U_{2}$ is $T(U_{1},U_{2})(\alpha)=\inf_{\zeta}\{\beta_{\zeta}:\alpha_{\zeta}\leq\alpha\}$ with $\zeta$ being a rejection rule, $\alpha$ being the type 1 error, and $\beta_{\zeta}$ the type 2 error for $\zeta$.

In our main Theorem 4.3, we prove that the DP framework for functions as defined in Theorem A.2 is $\mu-$GDP; we utilize the following corollary to achieve this.

###### Corollary 4.2 ( Dong et al. ( 2019 ) ) .

A mechanism is $\mu-$GDP if and only if it is $(\epsilon,\delta(\epsilon))-$DP for all $\epsilon\geq 0$, where $\delta(\epsilon)\geq\Phi\left(-\frac{\epsilon}{\mu}+\frac{\mu}{2}\right)-e^{
\epsilon}\Phi\left(-\frac{\epsilon}{\mu}-\frac{\mu}{2}\right).$

Corollary 4.2 also appears in Balle and Wang () in the context of the calibrating the Gaussian mechanism, but in the presented form the corollary is more readily applicable for our setting.

###### Theorem 4.3 .

The mechanism $\tilde{h}(D)=h(D)+\sigma Z$ as defined in Theorem A.2 is $\mu$-GDP with $\sigma\geq\Delta/\mu$. Here $\Delta$ is the same global sensitivity as before.

###### Proof.

We need to show that the distribution induced by our Gaussian process in $\mathbb{H}$ of $\tilde{h}(D)$ is $\mu-$GDP by bounding the $\delta(\epsilon)$ in Theorem A.2 as in Corollary 4.2. The key here is the use of $\mathbb{H}$, as our function space is infinite-dimensional and hence it is not possible to define a measure analogous to that of the Lebesgue measure.

Mirshani et al. () provide the framework to define a useful density of $\tilde{h}(D)$ over $\mathbb{H}$, however they do not consider bounding the tail probabilities nor the privacy loss random variable as we do next. We first define the privacy loss random variable as $PL=\log\left[\frac{dP(D)}{dQ}/\frac{dP(D^{\prime})}{dQ}\right]$, where $Q$ is the probability measure induced by our noise $Z$ and $P(D)$ the family of measures of our mechanism $\tilde{h}(D)$. By construction we have that $h(D)$ is compatible with $Z$ and hence the above is well defined:

$$ $\frac{dP(D)}{dQ}(y)=\exp\left\{-\frac{1}{2\sigma}\left[\|h(D)\|_{\mathcal{H}}^ {2}-2T_{h(D)}(y)\right]\right\}.$ $$

Here $T_{h(D)}$ is a linear operator and thus $T_{h(D)}(\tilde{h}(D))$ is normally distributed.

From Mirshani et al. (), we have that $T_{h(D)}(\tilde{h}(D))-T_{h(D^{\prime})}(\tilde{h}(D))\sim N(0,\|h(D)-h(D^{
\prime})\|^{2}_{\mathcal{H}})$.
Using this, we can show that

$$ $\displaystyle\delta$ $\displaystyle:=P\left(PL\geq\epsilon\right)-e^{\epsilon}P\left(PL\leq-\epsilon\right)$ (1) $\displaystyle=\Phi\left(-\frac{\epsilon\sigma}{\Delta}+\frac{\Delta}{2\sigma} \right)-e^{\epsilon}\Phi\left(-\frac{\sigma\epsilon}{\Delta}-\frac{\Delta}{2 \sigma}\right)$ (2) $\displaystyle=\Phi\left(-\frac{\epsilon}{\mu}+\frac{\mu}{2}\right)-e^{\epsilon }\Phi\left(-\frac{\epsilon}{\mu}-\frac{\mu}{2}\right).$ (3) $$

The last equality is a reparametrization setting $\mu=\Delta/\sigma$ and the rest of the details are in the A.7.
∎

For facial radial curves and other disc-like surfaces, by construction, we need to sanitize many mean curves, so we require composition of a multitude of mechanisms with their respective budgets $\mu_{i}$’s. Composition in the GDP framework is “tight."
Given two mechanisms $\tilde{h}_{1}$, $\tilde{h}_{2}$ with privacy parameters $\mu_{1}$ and $\mu_{2}$, their composition $(\tilde{h}_{1},\tilde{h}_{2})$ is $\sqrt{\mu_{1}^{2}+\mu_{2}^{2}}$-GDP (Dong et al., , Corallary 3.3).

###### Definition 4.1 .

###### Corollary 4.2 ( Dong et al. ( 2019 ) ) .

###### Theorem 4.3 .

###### Proof.

## 5 Face Radial Curve Example

Here, we apply our proposed methods from sections 3 and 4.2 on data described in Sero et al. () and A.4. Figure 3 displays from left to right the same face with $J=$ 16, 27, and 80 face radial curves. Larger values of $J$ add definition to the face, but $J$ need not be too large to encapsulate the facial structures. Based on that, we represent the $n=1000$ faces with $J=23$ face radial curves; one could pick $J$ in a data driven way as well. We treat the $J$ sets of curves across the faces independently.

Figure: Figure 3: Left: The same face represented using 16, 27, and 80 face radial curves, respectively. Right: The $x$, $y$, and $z$ coordinate curves, respectively, for a particular face radial curve.
Refer to caption: x6.png

We note an important subtle strength in our construction. Each face radial curve has three coordinates, $x,y,z$, and by our construction $x,y$ are effectively a circle and $z$ encode facial features. The right panel of Figure 3 displays the coordinate curves of an example face radial curve. The first two coordinates are quite simple and are roughly one period of a sine curve. We leverage this simplicity in these two coordinate curves to conserve privacy budget by enforcing a larger smoothing parameter as compared to the last coordinate curve and also treat the coordinate curves separately at each radial curve.

Let $\{f_{i}\}$ be the set of radial curves for a specific coordinate at the $j$th position and $i$ being the index of for particular face. For simplicity one can imagine $j=1$ being the curve nearest the tip of the nose and $j=23$ as the curve nearest the border of the face. Each curve $f_{i}$ is a closed parameterized curve, $f:\mathbb{S}^{1}\rightarrow\mathbb{R}$ where $\mathbb{S}^{1}$ is the unit circle, as in the right panels of Figure 3. In general, we drop the $j$ as we treat each set independent of each other. We parameterize the curve with unit circle but for simplicity we say $f:[0,1]\rightarrow\mathbb{R}$ with $f(0)=f(1)$.

By design we have closed curves, so to retain this structure we use a periodic kernel that takes the form $k(s,t)=\exp\left(-\left[d_{\mathbb{S}^{1}}(\omega(t),\omega(s))/\rho\right]^{
\alpha}\right)$ (Gneiting, ) with $t,s\in[0,1]$. The distance of two points $a,b$ on $\mathbb{S}^{1}$ is $\arccos\langle a,b\rangle$, however, the previous $t,s$ are parameters on the unit interval, so we first “wrap" the interval around the circle to compute this distance hence the need for $\omega(\cdot)$, a wrapping function. This kernel is a powered exponential on spheres with kernel range parameter $\rho$ and smoothness parameter $\alpha\in(0,1]$. For our experiments we set $\alpha=1$ as we can control smoothness instead via $\phi$ in the mean estimation as in 4.1.

We compute kernel matrix $K$ of the the unit interval $[0,1]$ with $[K]_{ij}=k(\omega(i/m),\omega(j/m))$ where $m=80$ is an integer defining the fineness of the uniform grid on the unit interval and $i,j=0,1,\dots m$. Let the eigenvalues, of $K$, and their associated eigenfunctions be denoted as $(\lambda_{i},b_{i})$. It can be shown that non-private mean in the RKHS space is $h(D)=\frac{1}{n}\sum_{i}\sum_{j}\frac{\lambda_{j}}{\lambda_{j}+\phi}\langle f_
{i},b_{j}\rangle b_{j}$. Since we are sanitizing the coordinates independently, we have a smoothness parameter for each coordinate, $\phi_{x}$, $\phi_{y}$, and $\phi_{z}$. The left panel of Figure 4 displays the mean face constructed from mean face radial curves with $\phi_{x}=\phi_{y}=0.01$ and $\phi_{z}=0.005$.

Figure: Figure 4: Left: Two angles of the average face constructed of average face radial curves with $\phi_{x}=\phi_{y}=0.01$ and $\phi_{z}=0.005$. Right: Two angles of the point-wise mean.
Refer to caption: x10.png

### 5.1 Experimental Results

To highlight the difficulty of working with such complex data we present some preliminary results. We have 1000 faces each with 7150 points; we fully describe the data in A.4. The points are “registered", via the data collection method, across all the faces. Without this registration, the following would not be immediately feasible. Let $D=\{X_{i}\}$ and $X_{i}\in\mathbb{R}^{7150\times 3}$. We first compute the point-wise mean of the faces, $\bar{X}=1/n\sum_{i}X_{i}$ and display this in the right panel of Figure 4.

Suppose we have a total budget of $\mu_{T}$ and we wish to sanitize, independently, the 7150 points each of which has 3 coordinates. To split the budget evenly, each point’s coordinate is allocated $\mu_{p}:=\sqrt{\mu_{T}^{2}/(7150\cdot 3)}$ of the budget. For each point and each coordinate we compute sensitivity as $\Delta_{k,l}=\max_{i,j}|X_{i}[k,l]-X_{j}[k,l]|$. That is, among all faces this measures the variability at each point. We note that this sensitivity calculation does violate privacy, as it is data driven, and that an entirely private form of this calculation is strictly larger. That is, our sensitivity calculation is smaller, so a private method would lead to noisier estimates. We add noise to each coordinate of each point as $\bar{X}[k,l]+\Delta_{k,l}/\mu_{p}\cdot z$ where $z\sim\mathcal{N}(0,1)$. Figure 5 displays two angles of a point-wise private face in the left panel with total privacy budget $\mu_{T}=3$ with more results in A.8.

Next we apply our approach using face radial curves. We have $J$ many sets of curves, $\{f_{i}\}_{j}$ each of which has three coordinate curves, $f_{i}=(f_{ix},f_{iy},f_{iz})$.
We sanitize the regularized mean, independently, for each coordinate and each radial curve, using our mechanism which satisfies GDP with sensitivity is $\Delta^{2}\leq 4\tau^{2}/(n^{2}\phi)$. Earlier we saw that the $x$ and $y$ coordinate curves, at every $J$, are effectively one period of a sinusoidal curve, we take advantage of this construction to conserve budget. We thus spend less budget sanitizing the $x$ and $y$ coordinate curves and spend more on $z$ as they contain more feature information.

Figure: Figure 5: Left: Two angles of a private face sanitized with point wise Gaussian noise with total $\mu_{T}=2$. Right: Two angles of a differentially private face constructed from private mean face radial curves with $\mu_{T}=2.9961$ $\phi_{x}=\phi_{y}=0.01$ and $\phi_{z}=0.005$.
Refer to caption: x14.png

We need to determine the $\tau$, an upper bound on the norm of the curves, in our $\Delta$. We determine this $\tau$, in a similar way as the point-wise approach of computing sensitivity, through the data at each set of radial curves and each coordinate. That is, for each $j$ and at each coordinate $w$, $\tau_{w}=\max_{i}\|f_{i}(w)\|_{\mathbb{H}}$. Again, we have a sensitivity that is partially data driven. However we note since we use shape analysis to do our processing, all the faces are scaled to have unit surface area, and hence all the curves are scaled. Thus, individually none of these norms hold any meaningful size information. Further, our analysis is entirely scale invariant, meaning that if a face is much larger or smaller than average, that information is completely removed when we scale. This is a strength of shape analysis as the emphasis is the shape of the face and not the nuisance parameters.

We choose a budget for each of the coordinates $\mu_{x},\mu_{y},\mu_{z}$ which are the same at all levels $j$ of curves; i.e., at each $j$ for the $w$ coordinate curve $\tilde{h}(\{f_{iw}\})=h(\{f_{iw}\})+\frac{\Delta_{w}}{\mu_{w}}Z$ with $\Delta_{w}^{2}\leq 4\tau_{w}w^{2}/(n^{2}\phi)$ and $Z$ is a standard normal Gaussian process. The right panel of Figure 5 displays an example private mean face with $\mu_{T}=2.9661$ where $\mu_{T}=\sqrt{J(\mu_{x}^{2}+\mu_{y}^{2}+\mu_{z}^{2})}$ with $J=23$, $\mu_{x}=\mu_{y}=0.2$ and $\mu_{z}=0.55$. For each coordinate we have separate smoothing parameters we display results with $\phi_{x}=\phi_{y}=0.01$ and $\phi_{z}=0.005$. Comparing our private mean to left panel of Figure 4 we see there is some clear smoothing in the lips area and some subtle smoothing in the eyes as well.

Lastly, we quantify the amount of injected noise by computing a mean squared error. We let the point-wise mean be our baseline non-private estimate, the right panel of Figure 5.
For the point-wise private estimate $\tilde{X}$ the $MSE=\frac{1}{N}\sum_{i}|\bar{X}_{i}-\widetilde{X}_{i}|^{2}$ where $N=7150$ is the number points. Table 1 displays these errors in the first two columns. Our sanitized mean, however, is a set of functions so we first discretize it into 1863 points (23 curves at 81 points) and denote this as $\widetilde{V}$. We let $MSE=\frac{1}{M}\sum_{j}\min_{i}|\bar{X}_{i}-\widetilde{V}_{j}|^{2}$ with $M=1863$. Since the two point clouds have different number of points, this MSE finds the nearest point in the non-private face to that of the private face $\widetilde{V}$. We further scale $\widetilde{V}\rightarrow a\widetilde{V}$ to align it to $\bar{X}$ as $\bar{X}$ is not processed data. Table 1 displays the error in the third column of 5.2989, less than both of the point-wise private faces. This MSE does incur an inflation, though, as the point-wise mean is not a surface and thus lacks a registered point at the location of our private mean. We can clearly see this in Figure 6, in both panels the blue points are the point-wise mean, the left panel has the point-wise private mean in red, and the right panel has our private (discretized) mean in red. We see that our method has points that seem to lie on the “surface" of the face but the non private mean may not have a point there. We also clearly see that the point-wise private mean adds noise in the ambient space and thus creates a rough estimate not entirely resembling a smooth face; i.e., the facial structures are distorted. Also, the MSE of our facial radial curve based GDP method is less than the MSE of the point-wise sanitized estimate which has a slightly larger privacy budget.
To even further emphasize this point, the last column of Table 1 is the MSE between our private mean and the RKHS mean, we see it is less than half in comparison to the point-wise sanitization.

**Table 1: Mean Squared Error between private estimates and the point-wise mean. The last column is the point-wise MSE between our method and the RKHS mean. All values are at E-04.**
|  | $\mu_{T}$ |  |  |  |
| --- | --- | --- | --- | --- |
|  | $2$ | $3$ | $2.9961$ (Ours) | Ours* |
| MSE | 14 | 7.5807 | 5.2989 | 3.6365 |

Figure: Figure 6: Blue points are the point-wise mean. Left: The red points are the point-wise private mean with $\mu_{T}=3$. Right: The red points are our private mean with $\mu_{T}=2.9961$.
Refer to caption: x18.png

## 6 Conclusions

We have developed a framework for releasing a $\mu$-GDP human face with our novel face radial curve representation. We extended $(\epsilon,\delta)$-DP FDA (Mirshani et al., ) techniques into the $\mu$-GDP framework (Dong et al., ) to take advantage of its tight composition of budgets. We utilized the new $\mu$-GDP FDA framework specifically for faces but note this is applicable for other FDA applications. Further, we utilize the shape analysis techniques of Samir et al. (); Drira et al. (); Jermyn et al. () to create a novel curve representation of a face. We focus on human faces, however, one can use our framework to sanitize other surfaces which are diffeomorphic to a disk.
Our representation changes the problem of sanitization from needing thousands of point-wise estimations to a few dozen functional estimates. We discuss the limitations of our methods in A.1.

A persons face contains information of ones identity and, as noted earlier, its image contains genomic information (Venkatesaramani et al., ), thus, the need to protect this information while sharing these data (e.g., anthropological studies) is crucial.
We demonstrate via empirical and quantitative results that our methodology adds less noise and preserves the structure of the face. We chose the number of face radial curves needed for our representation, but one could develop a private cross-validation method to do so.

## NeurIPS Paper Checklist

- 1.
Claims
- Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
- Answer: [Yes]
- Justification: The abstract has been written in a way to accurately reflect the content of the paper.
- Guidelines:
•
The answer NA means that the abstract and introduction do not include the claims made in the paper.
•
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
•
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
•
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
- 2.
Limitations
- Question: Does the paper discuss the limitations of the work performed by the authors?
- Answer: [Yes]
- Justification: We discuss the limitations of our methodology in the Supplementary material.
- Guidelines:
•
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
•
The authors are encouraged to create a separate "Limitations" section in their paper.
•
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
•
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
•
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
•
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
•
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
•
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
- 3.
Theory Assumptions and Proofs
- Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
- Answer: [Yes]
- Justification: All assumptions have been provided along with complete and correct proofs.
- Guidelines:
•
The answer NA means that the paper does not include theoretical results.
•
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
•
All assumptions should be clearly stated or referenced in the statement of any theorems.
•
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
•
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
•
Theorems and Lemmas that the proof relies upon should be properly referenced.
- 4.
Experimental Result Reproducibility
- Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
- Answer: [Yes]
- Justification: All tuning parameters have been provided and the authors have pointed towards locations of existing code that is used.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
•
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
•
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
•
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a)
If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b)
If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c)
If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
- 5.
Open access to data and code
- Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
- Answer: [No]
- Justification: We have provided GitHub links for much of the code which was used, however, the data is not yet publicly available.
- Guidelines:
•
The answer NA means that paper does not include experiments requiring code.
•
Please see the NeurIPS code and data submission guidelines ([https://nips.cc/public/guides/CodeSubmissionPolicy](https://nips.cc/public/guides/CodeSubmissionPolicy)) for more details.
•
While we encourage the release of code and data, we understand that this might not be possible, so “No” is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
•
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ([https://nips.cc/public/guides/CodeSubmissionPolicy](https://nips.cc/public/guides/CodeSubmissionPolicy)) for more details.
•
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
•
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
•
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
•
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
- 6.
Experimental Setting/Details
- Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
- Answer: [Yes]
- Justification: All choices are explicitly stated in their respective sections.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
•
The full details can be provided either with the code, in appendix, or as supplemental material.
- 7.
Experiment Statistical Significance
- Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
- Answer: [N/A]
- Justification: No error bars to report.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
•
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
•
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
•
The assumptions made should be given (e.g., Normally distributed errors).
•
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
•
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
•
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
•
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
- 8.
Experiments Compute Resources
- Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
- Answer: [Yes]
- Justification: All computational specifications have been stated in the Supplemental Materials.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
•
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
•
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).
- 9.
Code Of Ethics
- Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics [https://neurips.cc/public/EthicsGuidelines](https://neurips.cc/public/EthicsGuidelines)?
- Answer: [Yes]
- Justification: Yes, the research does indeed conform to the Code of Ethics.
- Guidelines:
•
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
•
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
•
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).
- 10.
Broader Impacts
- Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
- Answer: [Yes]
- Justification: The authors discuss the societal impacts of our work in the Introduction, Conclusions, and in the Limitations.
- Guidelines:
•
The answer NA means that there is no societal impact of the work performed.
•
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
•
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
•
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
•
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
•
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
- 11.
Safeguards
- Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
- Answer: [N/A]
- Justification: The data is not being released however the majority of the code used is already publicly available.
- Guidelines:
•
The answer NA means that the paper poses no such risks.
•
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
•
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
•
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
- 12.
Licenses for existing assets
- Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
- Answer: [Yes]
- Justification: All authors and owners of data and code have been adequately cited.
- Guidelines:
•
The answer NA means that the paper does not use existing assets.
•
The authors should cite the original paper that produced the code package or dataset.
•
The authors should state which version of the asset is used and, if possible, include a URL.
•
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
•
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
•
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, [paperswithcode.com/datasets](paperswithcode.com/datasets) has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
•
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
•
If this information is not available online, the authors are encouraged to reach out to the asset’s creators.
- 13.
New Assets
- Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
- Answer: [Yes]
- Justification: All new assets have been documented in the theoretical and experimental sections. Existing assets have their own documentation.
- Guidelines:
•
The answer NA means that the paper does not release new assets.
•
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
•
The paper should discuss whether and how consent was obtained from people whose asset is used.
•
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
- 14.
Crowdsourcing and Research with Human Subjects
- Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
- Answer: [Yes]
- Justification: We emphasize we did not collect any data yet we point to and describe briefly the data collecting method and the instructions human subjects were given at the time of data collection.
- Guidelines:
•
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
•
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
•
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
- 15.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
- Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
- Answer: [Yes]
- Justification: Again, we did no data collection but point to data collection method papers. The authors did receive IRB approvals to work with the collected data.
- Guidelines:
•
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
•
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
•
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
•
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

## Appendix A Supplemental Material

### A.1 Limitations

We do not collect any data, however, in A.4 we describe the data and how it was collected. In many regards our data is clean, so we did not test our methods on noisy data. In a sense, we can consider our data as processed data, so noisy data would require some processing which we do not consider in this work. It is possible that if the data is more noisy then it would require further smoothing which our method does allow. We further have an assumption of a genus-0 surface (a face with no holes) which may be an issue if there are missing data points. This type of data would require additional pre-processing. We also only test our method on one dataset; one could encounter the issue of variability of noise in other datasets but this can be overcome with the smoothness parameters in our method.

A limitation which we have already mentioned in 5.1 is the method for computing sensitivity, $\Delta$, in both our method and the point-wise method which we compare ourselves to. This, however, is an inherent issue in differential privacy rather than our method. That is, for both our method and the point-wise private estimate, we use the data to estimate the sensitivity. Ideally one could use a second dataset or a training set to estimate the many sensitivity values needed in a private way.

In terms of limitations of privacy and fairness, these are the exact problems we address. As mentioned in 1, one may want to release private average faces for demographics. Our methods are intended for facilitate this goal although we do not consider variability of sample sizes by demographics.

### A.2 Experiments Compute Resources

We ran experiments almost exclusively on a desktop computer with an Intel i7 processor and 32GB of RAM on Windows 11. Creating a triangulated mesh for a face takes about 5 seconds on the desktop, so this was done locally for each face. Parameterizing each face costs about 20 seconds on the desktop, this was also done locally. Reparameterizing, however, each face to a template costs upwards of 3 minutes on the desktop, so since we have 1000 faces we did this on a supercomputer.

Computing the mean and privatization with our method costs about 3 total minutes on the desktop computer. The mean computation and privatization of the point-wise method only requires about 1 minute of computational time on the desktop.

### A.3 Reparameterization

To optimally register two surfaces we use the methods as described in Jermyn et al. [] which we summarize next.
Suppose we have two surfaces $f_{1},f_{2}\in\mathcal{F}$ which we wish to optimally register over the parameterization group $\Gamma$.
Here $\mathcal{F}$ is as in 2.2 $\mathcal{F}\ni f:D\rightarrow\mathbb{R}^{3}$ where $D$ is the unit disk.
The action of $\Gamma$ on a surface is right composition, $f\circ\gamma$ for $\gamma\in\Gamma$, and reparameterizes the surfaces but does not change its image which in our case is the face. The authors in Jermyn et al. [] leverage a transformation of $f$ referred to as the square-root normal field (SRNF) defined as $q=\frac{n}{|n|^{1/2}}$ where $n$ is the normal vector $n=\frac{\partial f}{\partial u}\times\frac{\partial f}{\partial v}$. The corresponding action of $\Gamma$ on $q$ is then $(q,\gamma):=\sqrt{J(\gamma)}q\circ\gamma$. This transformation is necessary for an isometric action of $\Gamma$ but the details of this are lengthy and not necessary for our application.

The authors define an energy function $E_{reg}:\Gamma\rightarrow\mathbb{R}_{\geq 0}$ to implement an iterative gradient descent method for the registration. The energy is defined as $E_{reg}(\gamma)=\|q_{1}-(\tilde{q}_{2},\gamma)\|^{2}$ where $\tilde{q}_{2}$ is the “current" stage of the surface being reparameterized.
Here $\gamma$ is the incremental reparameterization with
$\tilde{q}_{2}=({q}_{2},\gamma)$.
Here $q_{1}$ would be our template surface from 2.2 and we register all surfaces to this template.

Since the reparameterization is done in an iterative manner it results in incrementally improved registration. This method is iterative as the gradient is taken about the identity of $\Gamma$, $\gamma_{identity}$ and thus lives in the tangent space of this element.
Suppose we have an orthonormal basis $\mathcal{B}=\{b\}$ for the tangent space of the $\Gamma$ at the identity, $T_{id}(\Gamma)$ where each $b$ be a unit vector in the tangent space $T_{id}(\Gamma)$.
The full gradient is given by $\sum_{b_{i}\in\mathcal{B}}\langle q_{1}-\tilde{q}_{2},d(b_{i},\gamma)\rangle_{
2}b_{i}$ and we use the implementation mentioned in 2.2 and refer the interested reader to the cited materials.

This reparameterization is costly due to both the dimension, iterative construction, and reliance on a basis. To alleviate some expense, we set the center of the surface as the tip of the nose as noted in A.6 with the triangulated mesh before we impose the disk parameterization. The reparameterization iterative method thus has to search over a smaller space as the center of the disk is pre-registered.

### A.4 Data

For a full description of how the data is collected we refer to Sero et al. [] but we summarize and emphasize the relevant points. All participants have the same neutral face expressions, so the data is not heterogeneous in terms of facial expression. Each face is captured with 7150 points which the authors refer to as quasi-landmarks. Further, these landmarks are registered across all individuals which Sero et al. [] refer to as “homologous." This registration is a point-wise correspondence across faces while the registration in A.3 is an entire disk correspondence. Similar to our approach the faces the faces are aligned using Procrustes Analysis, which finds the optimal rotation but againt considering the faces as a set of points not a disk-like surface.

### A.5 Supplemental Notes on DP

We first present the definition for approximate differential privacy.

###### Definition A.1 ( [Dwork et al., 2006 ] ) .

Let $D\sim D^{\prime}$ and $\tilde{h}(D)$ be a random privacy mechanism, the mechanism is said to achieve approximate differential privacy, $(\epsilon,\delta)$-DP, for some $\epsilon>0$ and $0<\delta<1$, if it satisfies the probabilistic inequality

$$ $P(\tilde{h}(D)\in A)\leq e^{\epsilon}P(\tilde{h}(D^{\prime})\in A)+\delta,$ $$

for any measurable set $A$.

Both $\epsilon$ and $\delta$ are pre-specified parameters referred to as the privacy budget. When $\delta=0$, this is referred to as pure differential privacy. Differential privacy is an attribute of the random mechanism and roughly states that the distributions over adjacent datasets are not too different. To achieve approximate DP for functional data Mirshani et al. [] establish the following mechanism.

###### Theorem A.2 ( Mirshani et al. [ 2017 ] ) .

Let $h(D)$ be a functional summary of a dataset $D$ that is compatible with standard Gaussian process noise $Z$ and $\epsilon\leq 1$. We have that $\tilde{h}(D):=h(D)+\sigma Z$ achieves $(\epsilon,\delta)-$DP over $\mathbb{H}$ where $\sigma\geq\frac{2\log(2/\delta)}{\epsilon^{2}}\Delta^{2}$. Here $\Delta$ is the global sensitivity of the summary $h(D)$ and $\Delta^{2}=\text{sup}_{D\sim D^{\prime}}\|h(D)-h(D^{\prime})\|_{\mathcal{H}}^{2}$ where the norm is over the space of the noise $Z$.

Assuming compatibility, let $Q$ denote the probability measure induced by $Z$ over $\mathbb{H}$ and $\{P(D):D\in\mathcal{D}\}$ denote the family of measures over $\mathbb{H}$ induced by $\tilde{h}(D)$ as in Theorem A.2. The density of $\tilde{h}(D)$ over $\mathbb{H}$ with respect to $Q$, which exists if compatability holds, takes the form

$$ $\frac{dP(D)}{dQ}(y)=\exp\left\{-\frac{1}{2\sigma}\left[\|h(D)\|_{\mathcal{H}}^ {2}-2T_{h(D)}(y)\right]\right\},$ $$

$Q$ almost everywhere with $T_{h(D)}(y)=\langle h(D),y\rangle_{\mathcal{H}}$.
The inner product on $\mathcal{H}$ can be defined in terms of eigenvalues ($\lambda_{i}$) and eigenfunctions ($b_{i}$), as $\langle x,y\rangle_{\mathcal{H}}=\sum_{i}\lambda_{i}^{-1}\langle x,b_{i}
\rangle_{\mathbb{H}}\langle y,b_{i}\rangle_{\mathbb{H}}$ but can generally be expressed using any basis.

###### Definition A.1 ( [Dwork et al., 2006 ] ) .

###### Theorem A.2 ( Mirshani et al. [ 2017 ] ) .

### A.6 Möbius Transformation

The reparameterization defined in 2.1 is computationally expensive. To make the repameterization less expensive, we leverage the Möbius transformation of conformal maps. We give a high level idea of this transformation with more details available in Choi and Lui [, ], Choi et al. [] and implementation available at GitHub repository Choi [].

The left most panel of Figure 7 displays a triangulated mesh of a face. The next step in our pre-processing is to generate a disk conformal map. The middle column of Figure 7 displays two different disk conformal maps of the same triangulated mesh in the left panel. It is, admittedly, difficult to discern features, but in each panel of the middle column one can see approximately four dense areas of which the middle corresponds to the nose. Recall that this disk conformal map embeds the triangulated mesh onto the disk while attempting to locally preserve all angles.

The panels in the right column of Figure 7 are the disk parameterized surface corresponding to the adjacent middle column disk conformal map. The center of the disk conformal disk, and hence the disk parameterized surface, is not known a priori. Either of these disk parameterized surfaces will suffice in our construction as we can optimally register it to a template as explained in 2.2. However, since reparameterization is costly, we can save cost by prespecifying the center of the disk of both the template and each face. A natural choice is to designate the center of the disk with feature such as the tip of the nose.

The bottom panel of the middle row is constructed using the Möbius transformation on the disk conformal map of the top panel of the middle row. We note that the Möbius transformation is for the disk conformal map and not the disk paramterized surface. This Möbius transformation costs effectively no time, but reparameterization of the bottom right disk parameterized surface is much faster than reparameterizing the top right to the template. This is because the methodology in A.3 looks at the gradient about the identity, so shifting the center of the disk requires a lot of energy. Having prespecified the nose as the center of the disk, though, the search space for reparamterization is much smaller. We lastly also note that all from the left and right columns of the figure have the same shape but have different representations; that is to say, shape is not effected by its parameterization nor representation.

Figure: Figure 7: Left column: A triangulated mesh of a face. Middle column: Two disk conformal maps of the triangulated mesh on the left. Right column: The disk parameterized surface resulting from the adjacent middle column conformal map.
Refer to caption: x20.png

### A.7 Proof details

We include details to the proof of Theorem 4.3. First we show the bound on the upper bound on the privacy loss random variable.

$$ $\displaystyle P(PL\geq\epsilon)$ $\displaystyle=P\left(\log\left[\frac{\exp-\frac{1}{2\sigma}(\|h(D)\|_{\mathcal {H}}^{2}-2T_{D}(x))\}}{\exp-\frac{1}{2\sigma}(\|h(D^{\prime})\|_{\mathcal{H}}^ {2}-2T_{D^{\prime}}(x))\}}\right]\geq\epsilon\right)$ $\displaystyle=P\left(\log\left[\exp\{-\frac{1}{2\sigma}(\|h(D)\|_{\mathcal{H}} ^{2}-\|h(D^{\prime})\|_{\mathcal{H}}^{2}-2(T_{D}-T_{D^{\prime}})(x)\}\right] \geq\epsilon\right)$ $\displaystyle=P\left(-\frac{1}{2\sigma^{2}}(\|h(D)\|_{\mathcal{H}}^{2}-\|h(D^{ \prime})\|_{\mathcal{H}}^{2}-2(T_{D}-T_{D^{\prime}})(x)\geq\epsilon\right)$ $$

Note that we have $\|h(D^{\prime})\|_{\mathcal{H}}-\|h(D)\|_{\mathcal{H}}=\|h(D)-h(D^{\prime})\|_
{\mathcal{H}}-2\langle h(D)-h(D^{\prime}),h(D)\rangle_{\mathcal{H}}$ and $\langle h(D)-h(D^{\prime}),h(D)\rangle_{\mathcal{H}}=(T_{D}-T_{D^{\prime}})(h(
D))$. Thus it follows that,

$$ $\displaystyle P(-\frac{1}{2\sigma^{2}}(\|h(D)\|_{\mathcal{H}}^{2}-\|h(D^{ \prime})\|_{\mathcal{H}}^{2}-2(T_{D}-T_{D^{\prime}})(x)\geq\epsilon)$ $\displaystyle=P(-\frac{1}{2\sigma^{2}}(-\|h(D)-h(D^{\prime})\|_{\mathcal{H}}^{ 2}-2(T_{D}-T_{D^{\prime}})(x-h(D))\geq\epsilon)$ $\displaystyle\leq P(-\frac{1}{2\sigma^{2}}(-\Delta^{2}-2(T_{D}-T_{D^{\prime}}) (x-h(D))\geq\epsilon)$ $\displaystyle=P((T_{D}-T_{D^{\prime}})(x-h(D))\geq\sigma^{2}\epsilon-\frac{ \Delta^{2}}{2})$ $\displaystyle=P(\sigma\Delta Z\geq\sigma^{2}\epsilon-\frac{\Delta^{2}}{2})$ $\displaystyle=P(Z\geq\frac{\sigma\epsilon}{\Delta}-\frac{\Delta}{2\sigma})= \Phi(-\frac{\sigma\epsilon}{\Delta}+\frac{\Delta}{2\sigma})$ $$

This is the upper bound we needed of the privacy loss random variable. We similar need a lower bound on the privacy loss random variable. The steps are very similar, we have that,

$$ $\displaystyle P(PL\leq\epsilon)$ $\displaystyle=P\left(\log\left[\frac{\exp-\frac{1}{2\sigma}(\|h(D)\|_{\mathcal {H}}^{2}-2T_{D}(x))\}}{\exp-\frac{1}{2\sigma}(\|h(D^{\prime})\|_{\mathcal{H}}^ {2}-2T_{D^{\prime}}(x))\}}\right]\leq\epsilon\right)$ $\displaystyle=P(Z\leq\frac{\sigma\epsilon}{\Delta}-\frac{\Delta}{2\sigma})= \Phi(\frac{\sigma\epsilon}{\Delta}-\frac{\Delta}{2\sigma}).$ $$

This completes the proof.

### A.8 Additional Results

In 5.1 we describe how to privatize a point-wise average face. Figure 8 displays a comparison of two private faces with $\mu_{T}=2$ and $\mu_{3}$, each from two angles. The two faces look fairly similar, but this is an artifact of the noise making the face rough and fuzzy. The face on the right column has a smoother outline as compared to the face on the left. In Table 1 we see that the MSE of the face on the right is nearly half of that of the face on the left.

Figure: Figure 8: Left: A private face sanitized with Gaussian noise point-wise with total $\mu_{T}=2$. Right: A private face sanitized with Gaussian noise point-wise with total $\mu_{T}=3$.
Refer to caption: x25.png

## Peer Reviews

confidence: 2
contribution: 2
initial_rating: 5
presentation: 1
questions: Please refer to the weaknesses.
soundness: 3
strengths: 1. The paper presents a novel approach to 3D face representation for privacy preservation using face radial curves, which is innovative and highly relevant to privacy concerns in biometric data.

2. The GDP mechanism integrated into this method ensures that facial features are preserved with minimal noise injection, enhancing privacy while maintaining utility.

3. The manuscript includes thorough experiments comparing the proposed method with traditional point-wise differential privacy techniques, demonstrating superior noise reduction and structure preservation.
summary: This paper introduces a method for creating Gaussian Differentially Private (GDP) representations of 3D human faces using a novel face radial curve representation. The proposed approach aims to address privacy concerns associated with sharing 3D facial data by employing differential privacy (DP) mechanisms. This method leverages statistical shape analysis to represent faces in a disk-parameterized structure, ensuring minimal noise addition while preserving the facial structure. The approach is specifically designed for applications that involve facial data but could extend to other disk-like surface data. The paper also presents experimental results demonstrating that the method injects less noise compared to point-wise differential privacy approaches while effectively preserving facial features.
weaknesses: 1. **Manuscript Organization**: The paper would benefit from a more structured organization aligned with the standard conference paper layout in AI. The current structure is challenging to follow, and the clarity could be enhanced by including diagrams that illustrate the overall pipeline of the proposed method. Guiding sentences that introduce and connect sections and paragraphs would also help readers navigate the content more effectively.

2. **Assumptions on Data Structure**: The proposed approach assumes a genus-0 surface with no missing data points, which may not be realistic in practical settings. This limitation necessitates additional pre-processing for noisy or incomplete data. Expanding the discussion on how the method could handle such cases would improve its practical applicability.

3. **Limited Dataset and Generalizability**: The evaluation is conducted on a limited dataset without a detailed description of its statistics, raising concerns about the method’s generalizability. While the authors suggest that this approach could extend to applications such as terrain models, no experiments support this claim. More comprehensive testing, including diverse or noisy datasets, is recommended to validate the method’s broader applicability.

4. **Computational Efficiency and Benchmarking**: Although the algorithm's methodology is well-described, the computational cost is not evaluated, which limits the ability to assess its practical performance. Additionally, the benchmark methods used for comparison are not clearly described, which could be clarified to provide a more comprehensive context for evaluating the proposed approach.

---

confidence: 2
contribution: 2
initial_rating: 3
presentation: 2
questions: Does the contributions of this paper concern the extension of GDP to functional data analysis, or the application to 3D face protection?
In Appendix B.3 refers to the SRNF  representation of surfaces but how this representation is used in this paper ?
soundness: 2
strengths: The idea of using Functional Data Analysis  Gaussian  mechanism  Differentially Private to protect 3D face is interesting.
summary: This paper proposes to extend  existing approximate Differentially Private (DP) Functional Data analysis (FDA) tools into the Gaussian DP framework. These idea have been applied to the protection of 3D face data, where each face is represented by a collection of a set of curves.
weaknesses: - The writing of this paper should be improved,  it does not facilitate the understanding of the contributions in this paper. 
- The approximation of 3D face by a set of radial curves has been published ins several papers related to 3D face recognition.  Hassen (1) Drira, Boulbaba Ben Amor, Mohamed Daoudi, and Anuj Srivastava. Pose and expression invariant
3d face recognition using elastic radial curves. In British machine vision conference, pp.
1–11, 2010. (2) Chafik Samir, Anuj Srivastava, and Mohamed Daoudi. Three-dimensional face recognition using
shapes of facial curves. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(11):
1858–1863, 2006.
- The contributions of this paper are not clear. Does the contributions of this paper concern the extension of GDP to functional data analysis, or the application to 3D face protection?
- The authors talk about functional data analysis which is a very general term, they should be more precise and discuss the representation of 3D faces by functions?
- In Appendix B.3 refers to the SRNF  representation of surfaces but how this representation is used in this paper ?

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: The paper needs to provide additional evidence to substantiate the usability of the proposed representation.
soundness: 3
strengths: (1) This paper proposes a novel privacy-preserving representation method for 3D faces. According to the authors, this approach is also applicable to other disk-like surfaces.
(2) The paper presents the mean squared error between private estimates and the point-wise mean, showing that this method outperforms the conventional point-wise method.
summary: The paper introduces a new representation termed "face radical curves" for a set of 3D faces. It then utilizes the Gaussian Differential Privacy (DP) framework on this representation to create a private average face. The empirical and quantitative results from the experiments show that this method not only maintains the shape of the average face but also introduces less noise compared to traditional methods for the same privacy budget.
weaknesses: （1) While the article effectively demonstrates the privacy capabilities of the representation through mean squared error metrics for 3D faces, it does not address the usability of the representation for practical applications, such as age estimation and expression analysis of faces. This aspect of usability verification is missing from the paper. 
（2）The authors suggest that the method described in the paper could be applicable to domains beyond 3D facial data. It would strengthen the paper if the authors could include experimental results demonstrating this method's effectiveness in other applications as well.

Meta-review: The paper introduces a Gaussian Differentially Private (GDP) method for 3D facial data using a novel radial curve-based representation to preserve privacy while maintaining structural fidelity. This paper proposes an innovative approach to integrating GDP with facial representations, and experimental validation showing superior performance in maintaining facial features. Before the rebuttal, the reviewers raise questions about unclear manuscript organization, limited dataset generalizability, and missing evidence of applicability to broader use cases. During the rebuttal, the authors have partially solved the questions. Thus I recommend accepting this paper.

Decision: Accept (Poster)