# Separation of the orbits in representations of SO₂ and O₂
over R and C

Martin Jalard*

Inria Côte d'Azur, France

November 4, 2024

## Abstract

I provide a minimal set of invariant polynomials separating the orbits for representations of SO₂ and O₂ over C and R. The idea is to select only polynomials of support of size 2 for SO₂ and 4 for O₂. I thus obtain cardinalities in respectively $O(\dim(V)^2)$ and $O(\dim(V)^4)$. These cardinalities are much smaller than for generating sets, which require polynomials of arbitrary large supports. Yet a separating set is sufficient for most of the applications. It appears also that real separating sets are smaller than the complex ones, which helps significatively for applications over R. I finally use the obtained separating set to stratify the real representations by isotropy classes.

<footnote>*Contact: martin.jalard@inria.fr</footnote>
# Contents

<table><tr><td>1 Introduction</td><td>2</td></tr><tr><td>2 Irreducible representations of SO<sub>2</sub> and O<sub>2</sub></td><td>3</td></tr><tr><td>3 Arithmetics and wheels</td><td>4</td></tr><tr><td>4 Separating invariants for SO<sub>2</sub>(C)</td><td>6</td></tr><tr><td>4.1 With rational invariants</td><td>6</td></tr><tr><td>4.2 With polynomial invariants</td><td>7</td></tr><tr><td>5 Separating invariants for SO<sub>2</sub>(R)</td><td>9</td></tr><tr><td>6 Separating invariants for O<sub>2</sub>(R)</td><td>10</td></tr><tr><td>6.1 Computation of the set.</td><td>10</td></tr><tr><td>6.2 Minimality</td><td>13</td></tr><tr><td>6.3 Comparison to a generating set</td><td>14</td></tr><tr><td>7 Separating invariants for O<sub>2</sub>(C)</td><td>15</td></tr><tr><td>7.1 Computation of the set.</td><td>15</td></tr><tr><td>7.2 Minimality</td><td>17</td></tr><tr><td>8 Stratification of real representations by orbit type</td><td>18</td></tr></table>

# 1 Introduction

I present minimal separating set for arbitrary representations of SO₂ and O₂ over $\mathbb{C}$ (Theorems 4.4 and 7.4), and over $\mathbb{R}$ (Theorems 5.2 and 6.5). Over the complex numbers, it means that the set separates closed orbits. Over the reals, it means more naturally that it separates all the orbits. The obtained sets have cardinalities in $O(N^2)$ for SO₂($\mathbb{R}$), $O(N^3)$ for SO₂($\mathbb{C}$) and $O(N^4)$ for O₂($\mathbb{R}$) and O₂($\mathbb{C}$), where N is the dimension of the representation.

Quite few general results provide separating sets in invariant theory, the most common approach being to compute a set generating the invariant algebra. Nevertheless, the first can have much smaller cardinalities. Yet they still provide an epimorphism to the orbit space, so that they replace successfully generating sets for many applications. For instance, they can be used to stratify the representation by orbit type. A complete stratification of representations of SO₂($\mathbb{R}$) and O₂($\mathbb{R}$) is provided in Section 8. Separating set are also sufficient to compute universal invariant or equivariant networks converging on any compact [DG24; BV23]. The size of the separating set corresponds to the dimension of the first layer, thus the cardinality heavily reflects on the complexity.

The idea to obtain small separating sets for SO₂($\mathbb{C}$) is to select only monomials of minimal support. A representation of the torus SO₂($\mathbb{C}$) $\cong \mathbb{C}^*$ on $\mathbb{C}^N$ is determined by a row matrix $A \in \mathbb{Z}^N$. The invariant algebra is generated by monomial corresponding to the irreducible solutions of $\{e \in \mathbb{N}^N, Ae = 0\}$ [Stu08]. Many large support in $\{1, ..., N\}$ provide irreducible monomials, thus their cardinality is up to $2^N$. Whereas, rational monomials of support of cardinality two are sufficient to separate the orbits. This comes from the arithmetical key Lemma 3.4. I next deduce a set of separating polynomials of support of size at most three. And some polynomials of support at most four are sufficient to separate orbits of O₂($\mathbb{C}$). Thus the cardinalities in respectively $O(N^2)$, $O(N^3)$ and $O(N^4)$ $\ll 2^N$. Furthermore, these separating sets are provided by explicit expressions whereas generating sets are computed by heavy algorithms [Stu08; Des+23].
One notices that for $SO_2(\mathbb{C})$, rational invariants are particularly efficient to separate orbits. The rational separating set has a lower cardinality compared to the polynomial one ($O(N^2) \ll O(N^3)$). The difference appears clearly in Example 4.7. Furthermore, rational invariants separate most of the non closed orbits, which is not possible with polynomial invariants. [Bür+21] also pointed out the relevance of rational invariants to separate orbits of toric actions. But the algorithms it proposes compute invariants for any support in $[1, N]$. A total set of separating invariants would be much too large to be computed by that way.

Since the definition of separation is weaker over $\mathbb{R}$, I also obtain smaller real separating sets (again of cardinality in $O(N^2) \ll O(N^3)$ for $SO_2(\mathbb{R})$). [Duf09] already notices this difference for some finite groups. Still this phenomenon has not been developed that much in invariant theory. This is challenging, most of the applications concerning more particularly the real numbers.

For a summand of standard representations, the separating sets of this article coincides with Weyl invariants [Wey46, Theorem 2.9A] which are at the same time minimal generating and separating sets (see Remarks 5.3 and 6.9). I thus generalize this result to summand of arbitrary representations. The irreducible representations of $SO_2$ and $O_2$ being indexed by $\mathbb{N}$, they induce arithmetic difficulties.

If one has a separating set for a reductive group, almost all set of $2N + 1$ linear combinations of the polynomials is a separating set [DK15]. But as discussed in [Kem24, Remark 6.2] it still is relevant to compute minimal separating sets of higher cardinal. Indeed the linear combinations remove powerful properties of the polynomials (the homogeneity or the low dimension of the supports).

The paper is organized as follows. I start establishing essential statements about irreducible representations of $SO_2$ and $O_2$ in Section 2. I next give the key arithmetical Lemmas which allows to select monomials of support two in Section 3. Resulting separating sets are presented in Sections 4, 5, 6 and 7. I finish with the stratification of real representations by orbit type in Section 8. For a real vector space $\mathcal{V}$, I note $\hat{\mathcal{V}} = \mathcal{V} \oplus i\mathcal{V}$ its complexification.

## 2 Irreducible representations of $SO_2$ and $O_2$

We first establish the real and complex irreducible representations of $SO_2$ and $O_2$. There is a semidirect product $O_2(\mathbb{C}) \cong SO_2(\mathbb{C}) \times C_2$. For $z \in \mathbb{C}^*$ we note $g_z^+ = (z, 1) \in SO_2(\mathbb{C})$ and $g_z^- = (z, -1) \in O_2(\mathbb{C})$. As a torus, $SO_2(\mathbb{C})$ has irreducible representations of dimension one. These are gathered pairwise to form the irreducible representations of $O_2(\mathbb{C})$:

**Theorem 2.1.** The irreducible representations of $SO_2(\mathbb{C})$ are $(\phi_j, \mathcal{L}_j = \mathbb{C})$, $j \in \mathbb{Z}$ where for $z \in \mathbb{C}^*$,

$$ \phi_j(g_z^+) = z^j. $$

**Theorem 2.2.** [GSS88] The irreducible representations of $O_2(\mathbb{C})$ consist of

* $(\rho_i, \hat{\mathcal{V}}_i = \mathbb{C}^2)$ for $i \in \mathbb{N} \setminus \{0\}$, where for any $z \in \mathbb{C}^*$,

$$ \rho_i(g_z^+) = \begin{pmatrix} z^i & 0 \\ 0 & z^{-i} \end{pmatrix} \quad \text{and} \quad \rho_i(g_z^-) = \begin{pmatrix} 0 & z^{-i} \\ z^i & 0 \end{pmatrix}. $$

* The trivial representations $(\rho_0, \hat{\mathcal{V}}_0 = \mathbb{C})$, and the determinant $(\rho_{-1}, \hat{\mathcal{V}}_{-1} = \mathbb{C})$.

The standard parameterization of $SO_2(\mathbb{R})$ is $\{ \begin{pmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \end{pmatrix}, \theta \in \mathbb{R} \}$. The conjugation by $P = \begin{pmatrix} 1 & i \\ 1 & -i \end{pmatrix}$ recovers the parameterization

$$ O_2(\mathbb{R}) \cong \left\{ g_z^+ = \begin{pmatrix} z & 0 \\ 0 & z^{-1} \end{pmatrix}, z \in \mathcal{U} \right\} \cup \left\{ g_z^- = \begin{pmatrix} 0 & z^{-1} \\ z & 0 \end{pmatrix}, z \in \mathcal{U} \right\}, $$

where $\mathcal{U} \subset \mathbb{C}$ is the unit circle. We obtain the irreducible representations restricting the complex representations to the real trace:
**Theorem 2.3.** The irreducible representations of SO₂($\mathbb{R}$) consist of

• $(\rho_i, \mathcal{V}_i)$, for $i \in \mathbb{N} \setminus \{0\}$.

• The trivial representation $(\rho_0, \mathcal{V}_0)$.

**Theorem 2.4.** The irreducible representations of O₂($\mathbb{R}$) are

• $(\rho_i, \mathcal{V}_i)$, for $i \in \mathbb{N}^*$.

• The trivial representation $(\rho_0, \mathcal{V}_0)$ and the determinant $(\rho_{-1}, \mathcal{V}_{-1})$.

For each $i$ we endow $\hat{\mathcal{V}}_i$ with coordinates $(v_{-i}, v_i)$. On the real trace $\mathcal{V}_i \subset \hat{\mathcal{V}}_i$, the coordinates satisfy $v_i = \bar{v}_{-i}$. Throughout the article we ignore the trivial representations. Indeed, to separate the orbits in a representation with an additional trivial part $\mathcal{L}_0^k$, it suffices to add the linear polynomials $\{t_i, 1 \le i \le k\}$.

# 3 Arithmetics and wheels

The results of the paper rely on some technical lemmas of arithmetic. They describe how complex wheels intersects. In particular, the idea to keep only invariants of support two results from Lemma 3.4 and its Corollary 3.5.

**Definition 3.1.** For some $c \in \mathbb{C}^*$ and $a \in \mathbb{Z} \setminus \{0\}$, we define $W_c^a$ the wheel of $a^{th}$ roots of $c$: $W_c^a := \{z \in \mathbb{C}^* | z^a = c\}$.

Let $u, v$ be two non zero vectors in the representation of SO₂($\mathbb{C}$) on $\mathcal{L}_a$ for some $a \in \mathbb{Z}^*$. Then, $W_u^a$ is the set of complex numbers $z$ such that $\phi_a(u) = v$. For two integers $v, v' \in \mathbb{Z}$, we note $v \wedge v'$ their greatest common divisor and $v \vee v'$ their least positive common multiple.

**Lemma 3.2.** Let $u, u', v, v'$ be integers and take the two set $U := u + r\mathbb{Z}$ and $U' := u' + r'\mathbb{Z}$. There is the equivalence:

$$U \cap U' \neq \emptyset \Leftrightarrow (v \wedge v') \text{ divides } (u - u')$$

*Proof.* The Bezout identity ensures that $r \wedge r'$ is the least positive integer which is a linear combination of $r$ and $r'$. Hence, for any integer $u, \exists k, k' \in \mathbb{Z}, u = kr' + k'r' \Leftrightarrow (r \wedge r') \text{ divides } u$. Then, $(r \wedge r') \text{ divides } (u - u')$ iff $\exists k, k' \in \mathbb{Z}, u - u' = kr + k'r'$. This is equivalently $\underbrace{u - kr}_{\in U} = \underbrace{u' + k'r'}_{\in U'}$. $\square$

**Corollary 3.3.** Let $c, c'$ be two complex numbers in $\mathbb{C}^*$ and $a, a'$ non zero integers. Then we have

$$W_c^a \cap W_{c'}^{a'} \neq \emptyset \Leftrightarrow c^{\frac{a \vee a'}{a}} = c'^{\frac{a \vee a'}{a'}}$$

In that case $W_c^a \cap W_{c'}^{a'} = W_{c'c^{a'}}^{a \wedge a'}$, where $q, q'$ are Bezout coefficients satisfying $aq + a'q' = a \wedge a'$.

<figure><img src="image_1.png" /><figcaption>Figure 1: Intersection of wheels</figcaption></figure>

*Proof.* Let $w$ be a root in $W_{c'}^{a'}$. Recall that $aa' = (a \vee a')(a \wedge a')$. Then, $w^{a \vee a'} = w^{\frac{aa'}{a \wedge a'}} = c^{\frac{a}{a \wedge a'}} = c^{\frac{a \vee a'}{a'}}$. Thus,

$$c^{\frac{a \vee a'}{a}} = c'^{\frac{a \vee a'}{a'}} \Leftrightarrow W_c^a \subset W_{c^{\frac{a \vee a'}{a}}}^{a \vee a'} \text{ and } W_{c'}^{a'} \subset W_{c^{\frac{a \vee a'}{a}}}^{a \vee a'}$$

Note $\varphi: \mathbb{Z} \to W_{c^{\frac{a \vee a'}{a}}}^{a \vee a'}$ a surjective map with, for any integer $k$, $\varphi(k) = \varphi(0) * \exp(i \frac{2\pi k}{a \vee a'})$. Then the preimage $\varphi^{-1}(W_c^a)$ is some $U = \{u + \frac{a \vee a'}{a}k, k \in \mathbb{Z}\}$ meanwhile the preimage $\varphi^{-1}(W_{c'}^{a'})$ is some $U' = \{u' + \frac{a \vee a'}{a'}k, k \in \mathbb{Z}\}$. However, $(\frac{a \vee a'}{a}) \wedge (\frac{a \vee a'}{a'}) = 1$. According to lemma 3.2, one has $U' \cap U \neq \emptyset$. This gives $W_c^a \cap W_{c'}^{a'} \neq \emptyset$, and one deduces that $W_c^a \cap W_{c'}^{a'} = W_z^{a \wedge a'}$ for some unknown complex $z \in \mathbb{C}^*$.
Nevertheless, for any $w \in W_c^a \cap W_{c'}^{a'} = W_z^{a \wedge a'}$ one has $c = w^a = w^{\frac{aa'}{a'}} = w^{(a \wedge a')}^{\frac{a \vee a'}{a'}} = z^{\frac{a \vee a'}{a'}}$ and $c' = w^{a'} = w^{\frac{aa'}{a}} = w^{(a \wedge a')}^{\frac{a \vee a'}{a}} = z^{\frac{a \vee a'}{a}}$. If one choose, thanks to Bezout identity, $q$ and $q'$ such that $aq + a'q' = a \wedge a'$, then $q^{\frac{a \vee a'}{a'}} + q'^{\frac{a \vee a'}{a'}} = 1$. That is, $z = z^{q^{\frac{a \vee a'}{a'}} + q'^{\frac{a \vee a'}{a}}} = c^q c'^{q'}$. $\square$

The following lemma is key in selecting invariants of low support:

**Lemma 3.4.** Let $U_1, \dots, U_n \subset \mathbb{Z}$ be subsets defined by $U_i = u_i + r_i\mathbb{Z}$ for some $u_i, r_i \in \mathbb{Z}$. Suppose that $\forall 1 \le i < j \le n$, $U_i \cap U_j \ne \emptyset$. Then, $\bigcap_{i=1}^r U_i \ne \emptyset$.

*Proof.* We proceed by induction on the number $n$. The lemma holds for $n^+ = 2$. Take $n^+ > 2$ such that the lemma holds for $n^+ - 1$. Then $\bigcap_{i=1}^{n^+-1} U_i \ne \emptyset$, and up to translation, we can suppose that $u_i = 0$ for $i \le n^+ - 1$.

Then, $\bigcap_{i=1}^{n^+-1} U_i = \{kM, k \in \mathbb{Z}\}$ where $M := \text{lcm}(r_1, \dots, r_{n^+-1})$.

By Lemma 3.2 we have $U_i \cap U_{n^+} \ne \emptyset \Leftrightarrow (r_i \wedge r_{n^+})$ divides $(u_n - u_i) = u_{n^+}$. For all prime number $p$ and integer $u$ we note $\nu_p(u) = \max\{k \in \mathbb{N} | p^k$ divides $u\}$ the $p$-adic valuation of $u$.

$$ \nu_p(u_{n^+}) \ge \max_{i \le n^+ - 1} [\nu_p(r_i \wedge r_{n^+})] = \max_{i \le n^+ - 1} [\min(\nu_p(r_i), \nu_p(r_{n^+}))] $$

If there is some $1 \le i \le n - 1$ such that $\nu_p(r_i) > \nu_p(r_n)$, then

$$ \max_{i \le n^+ - 1} [\min(\nu_p(r_i), \nu_p(r_{n^+}))] = \nu_p(r_n) = \min[\max_{i \le n^+ - 1} (\nu_p(r_i)), \nu_p(r_{n^+})]. $$

Otherwise $\nu_p(r_n) \ge \nu_p(r_i)$ for any $1 \le i \le n - 1$ and

$$ \max_{i \le n^+ - 1} [\min(\nu_p(r_i), \nu_p(r_{n^+}))] = \max_{i \le n^+ - 1} [\nu_p(r_i)] \ge \min[\max_{i \le n^+ - 1} (\nu_p(r_i)), \nu_p(r_{n^+})]. $$

It provides

$$ \nu_p(u_{n^+}) \ge \min[\max_{i \le n^+ - 1} (\nu_p(r_i)), \nu_p(r_{n^+})] = \nu_p(M \wedge r_n) $$

This is, $M \wedge r_{n^+}$ divides $u_{n^+}$. Hence, apply the Lemma 3.2 again to have $U_{n^+} \cap \bigcap_{i=1}^{n^+ - 1} U_i \ne \emptyset$. $\square$

**Corollary 3.5.** Let $\{W_i = W_{c_i}^{a_i}, i \in I\}$ be a finite set of wheels such that any pair of them has a non empty intersection: $\forall i, j \in I$, $W_i \cap W_j \ne \emptyset$. Then, $\bigcap_{i \in I} W_i \ne \emptyset$.

*Proof.* Take $\{W_i = W_{c_i}^{a_i}, i \in I\}$ such a set and note $M = \text{lcm}(a_i, i \in I)$. The union $\bigcup_{i \in I} W_i$ is contained in a big wheel $W$ of cardinal $M$. This wheel can be defined for instance as $W = W_{c_1}^{M}$, $W_{c_1}^{M}$ is a surjective map $\varphi: \mathbb{Z} \to W$ with, for any integer $k$, $\varphi(k) = \varphi(0) \exp(i \frac{2\pi k}{M})$. For $i \in I$, one has the preimage $U_i := \varphi^{-1}(W_i) = \{u_i + k \frac{M}{a_i}, k \in \mathbb{Z}\}$. For any $i, j \in I$, $W_i \cap W_j \ne \emptyset \Rightarrow U_i \cap U_j \ne \emptyset$. According to lemma 3.4 then, $\bigcap_{i \in I} U_i \ne \emptyset$. This conversely gives $\bigcap_{i \in I} W_i \ne \emptyset$. $\square$

Not all pairwise intersections are necessary to get an overall intersection. It depends on the values of the integers $a_i$, as it appears on figure 2 with 2, 3, -6.

**Proposition 3.6.** Let $W_1 = W_{c_1}^{a_1}$, $W_2 = W_{c_2}^{a_2}$ and $W_3 = W_{c_3}^{a_3}$ be three wheels. Suppose that $W_1 \cap W_2$ and $W_1 \cap W_3$ are non empty. If $a_1$ divides $a_2 \vee a_3$, then $W_2 \cap W_3 \ne \emptyset$.

<figure><img src="image_2.png" /><figcaption>Figure 2: Automatic intersection</figcaption></figure>
*Proof.* Define a surjective map $\varphi : \mathbb{Z} \to W_1$ with, for any integer $k$, $\varphi(k) = \varphi(0) \exp(i\frac{2\pi k}{a_1})$. One has the preimages $U_2 := \varphi^{-1}(W_1 \cap W_2) = \{u_2 + k\frac{a_1}{a_1 \wedge a_2}, k \in \mathbb{Z}\}$ and $U_3 := \varphi^{-1}(W_1 \cap W_3) = \{u_3 + k\frac{a_1}{a_1 \wedge a_3}, k \in \mathbb{Z}\}$. Lemma 3.3 claims that $(U_2 \cap U_3 \neq \emptyset) \Leftrightarrow$ (M divides $u_2 - u_3$), where $M = (\frac{a_1}{a_1 \wedge a_2} \wedge \frac{a_1}{a_1 \wedge a_3})$. A piece of p-adic calculus provides $M = \frac{a_1}{a_1 \wedge (a_2 \vee a_3)}$. Then $a_1 | (a_2 \vee a_3) \Leftrightarrow$ M = 1, and M divides $u_2 - u_3$. Otherwise we choose $c_1 = c_2 = 1$ and $c_3 = \exp(\frac{2i\pi a_3}{a_1})$. Then, $c_2 \wedge a_2 = c_3 \wedge a_3 = \exp(\frac{2i\pi a_3}{a_1} \frac{a_2 \vee a_3}{a_3}) = \exp(\frac{2i\pi a_2 \vee a_3}{a_1}) \neq 1$. Then lemma 3.3 claims that $W_2 \cap W_3 = \emptyset$. $\square$

# 4 Separating invariants for SO₂($\mathbb{C}$)

In this section we provide minimal separating set for arbitrary representations of SO₂($\mathbb{C}$). Let $a_1, ..., a_{n+}$ be positive integers and $a_0, ..., a_{-n-}$ negative integers. We consider the representation of SO₂($\mathbb{C}$) of dimension $N = n^- + n^+ + 1$

$$ \hat{\mathcal{V}} = \bigoplus_{i=0}^{n^-} \mathcal{L}_{a_{-i}} \oplus \bigoplus_{i=1}^{n^+} \mathcal{L}_{a_i} $$

and endow it with the coordinates $(v_{-n-}, ..., v_{n+})$. To a vector $e = (e_{-n-}, ..., e_{n+})$ in $\mathbb{Z}^N$ corresponds the rational monomial $m_e = \prod_{i=-n^-}^{n^+} v_i^{e_i}$. Consider the row matrix $A = (a_{-n-}, ..., a_{n+})$. Then the module $M_A = \{e \in \mathbb{Z}^N | A e = 0\}$ corresponds to the Laurent monomials in $\mathbb{C}(\hat{\mathcal{V}})^{SO_2(\mathbb{C})}$. The monoid $M_A^+ = \{e \in \mathbb{Z}^N | A e = 0\}$ corresponds to the monomials of positive degree in $\mathbb{C}(\hat{\mathcal{V}})^{SO_2(\mathbb{C})}$ [Stu08, Lemma 1.4.2]. One naturally defines the support of a monomial $m_e$ as the set of indices $i$ such that $e_i \neq 0$.

We first exhibit a set of cardinality $O(N^2)$ composed of rational monomials of support of size two separating any orbits with two non zero coordinates (Theorem 4.1). Next we deduce a polynomial set of cardinality $O(N^3)$ separating any closed orbits, by adding some monomials of support of size three (Theorem 4.4). The latter needs not be minimal for inclusion. We thus give a criterion to remove redundant polynomials (Proposition 4.5).

## 4.1 With rational invariants

As pointed out in [Bür+21], rational invariants of tori separate many not closed orbits, something that is not possible with any set of polynomial invariants. Furthermore, a very lower number of rational invariants allows to separate orbits. Here, we compute a separating set of cardinality $\frac{N(N+1)}{2} \sim N^2$.

Let $u, v$ be two points in $\hat{\mathcal{V}}$. We note $W_i$ the set of complex numbers $z$ such that $\phi_i(g_z^+)(u_i) = v_i$. When neither $u_i$ nor $v_i$ vanish, we note $c_i = \frac{v_i}{u_i}$, and then $W_i = W_{c_i}^{a_i}$ is the wheel of $a_i^{th}$ roots of $c_i$.

**Theorem 4.1.** For any $-n^- \le i < j \le n^+$, we note $\varepsilon_{ij} = \frac{a_i \vee a_j}{a_i}$. Consider the set of rational invariant

$$ f^* = \{ p_{ij} = v_i^{-\varepsilon_{ij}} \times v_j^{\varepsilon_{ji}}, -n^- \le i < j \le n^+ \} $$

and take two point $u$ and $v$ with at least two non zero coordinates. Then,

$$ \exists P \in f^*, P(u) \neq P(v) \Leftrightarrow u, v \text{ are not in the same orbit.} $$

*Proof.* Firstly for any pair of integers $-n^- \le i < j \le n^+$, one has $\frac{a_i \vee a_j}{a_i} - \frac{a_i \vee a_j}{a_j} a_j = 0$. Then $p_{ij}$ is invariant.

Take $v \in \mathcal{V}$ with two non zero coordinates. For any $-n^- \le i < j \le n^+$, $\begin{cases} v_i \neq 0 \\ v_j \neq 0 \end{cases} \Leftrightarrow p_{ij}(v) \neq 0$. Since there exists $j$ such that $v_j \neq 0$, the polynomials $\{p_{i-j}, -n^- \le j \le n^+\}$ separate orbits with $v_i = 0$ from orbits with $v_i \neq 0$. We can suppose then that none of the coordinates of $v$ vanishes.
Take $u$ another point without zero coordinate such that for any $P \in f^*$, $P(u) = P(v)$. For all $-n^- \le i < j \le n^+$, one has $p_{ij}(v) = p_{ij}(u) \Rightarrow \frac{p_{ij}(v)}{p_{ij}(u)} = c_i^{\varepsilon_{ij}} c_j^{-\varepsilon_{ji}} = 1$. Then, Corollary 3.3 claims that $W_i \cap W_j \neq \emptyset$. Corollary 3.4 ensures the existence of some non zero complex number $z$ in the intersection $\bigcap_{i=-n^-}^{n^+} W_i$. This number verifies $\rho(g_z^+)(u) = v$. $\square$

Note that rational invariants of support of size 1 do not exist. Hence, it is not possible to separate more orbits with rational invariants. As in [Bür+21], we separate the last orbits (of support 0 and 1) by their support. Two vectors of support of size less than 1 lie on the same orbit iff they have the same support.

## 4.2 With polynomial invariants

Functions $p_{ij}$ of Theorem 4.1 are not polynomial whenever $i$ and $j$ have the same sign. One may need a set of invariant polynomials. Because they are smooth, invariant polynomials cannot separate non closed orbits from their closure. There is thus the following definition of separation:

**Definition 4.2.** [DK15, Definition 2.3.8] Let $\hat{\mathcal{V}}$ be a complex $G$-variety. A subset $S \subset \mathbb{C}[\hat{\mathcal{V}}]^G$ is said to be separating if for any $u, v \in \hat{\mathcal{V}}$,

$$ \exists P \in \mathbb{C}[\hat{\mathcal{V}}]^G, P(u) \neq P(v) \Leftrightarrow \exists Q \in S, Q(u) \neq Q(v). $$

Since $\mathrm{SO}_2(\mathbb{C})$ is the complexification of the compact Lie group $\mathrm{SO}_2(\mathbb{R})$, it is a reductive group [Ser66, Theorem 5]. It is thus sufficient to find a set separating closed orbits [MFK94]. The following lemma determines the closed orbits of $\mathrm{SO}_2(\mathbb{C})$ in the representation $\hat{\mathcal{V}} = \bigoplus_{i=0}^{n^-} \mathcal{L}_{a_{-i}} \oplus \bigoplus_{i=1}^{n^+} \mathcal{L}_{a_i}$.

**Lemma 4.3.** A point $v \in \hat{\mathcal{V}}$ belongs to a closed orbit iff

$$ \exists 1 \le i \le n^+, v_i \ne 0 \Leftrightarrow \exists 0 \le j \le n^-, v_{-j} \ne 0 $$

*Proof.* Choose $v \in \mathcal{V}$ with some $1 \le i \le n^+$ satisfying $v_i \ne 0$. Assume that there exists a sequence $z_n \in \mathbb{C}^N$ such that $\rho(g_z^+)(v)$ converges in $\hat{\mathcal{V}}$. Since $\mathrm{SO}_2(\mathbb{C})$ acts regularly it implies that $z_n$ converges toward some $z \in \mathbb{C}$. If $z \ne 0$, $\rho(g_z^+)(v)$ converges toward $\rho(g_z^+)(v)$, and this limits belongs to the orbit of $v$. If $z = 0$, suppose that $\exists j \le n^-$ such that $v_j \ne 0$. Hence, $\frac{1}{z_n}v_j$ converges, and that is not possible.

Otherwise, if $\forall j \le n^-$, $v_{-j} = 0$, then $\rho(g_z^+)(v)$ converges to 0, and this limits does not belong to the orbit of $v$. Hence $v$ does not belong to a closed orbit. $\square$

Take $-n^- \le k \le 0 < i < j \le n^+$ or $-n^- \le i < j \le 0 < k \le n^+$. Then by Bezout identity, we choose $q, q'$ and $r, r'$ non negative integers such that $\begin{cases} a_i q + a_k q' = a_i \wedge a_k. \\ a_j r + a_k r' = -a_j \wedge a_k. \end{cases}$ With respect to this choice, we construct the polynomials

$$ h_{ijk} = v_i^{q \varepsilon_{ijk}} \times v_j^{r \varepsilon_{ijk}} \times v_k^{q' \varepsilon_{ijk} + r' \varepsilon_{ijk}} $$

where $\varepsilon_{ijk} = \frac{(a_i \wedge a_k) \vee (a_j \wedge a_k)}{a_i \wedge a_k}$.

**Theorem 4.4.** The following set of invariant polynomials separates closed orbits:

$$ f = \left\{ \begin{array}{ll} p_{ij}, & -n^- \le i < 0 \le j \le n^+ \\ h_{ijk}, & -n^- \le k \le 0 < i < j \le n^+ \text{ or } -n^- \le i < j \le 0 < k \le n^+ \end{array} \right\} $$

*Proof.* Firstly take $-n^- \le k \le 0 < i < j \le n^+$ or $-n^- \le i < j \le 0 < k \le n^+$. We compute the weighted sum of the exponents to prove that $h_{ijk}$ is invariant:

$$ \begin{aligned} a_i q \varepsilon_{ijk} + a_j r \varepsilon_{ijk} + a_k (q' \varepsilon_{ijk} + r' \varepsilon_{ijk}) &= (a_i \wedge a_k) \vee (a_j \wedge a_k) \left( \frac{qa_i + q'a_k}{a_i \wedge a_k} + \frac{ra_j + r'a_k}{a_j \wedge a_k} \right) \\ &= (a_i \wedge a_k) \vee (a_j \wedge a_k)(1-1) = 0 \end{aligned} $$
Let $u$ and $v$ be two points in closed orbits such that for any $P \in f$, $P(u) = P(v)$. Let $1 \le i < j \le n^+$ be indices such that $u_i, u_j \ne 0$. According to lemma 4.3, there also exists some $1 \le k \le n^-$ such that $u_k \ne 0$. Then, $p_{ik}(u) = p_{ik}(v)$ implies that $v_i, v_k \ne 0$ and $W_i \cap W_k \ne \emptyset$. Following lemma 3.3 we have $W_i \cap W_k = W_i^{a_i \wedge a_k}$, and similarly $W_j \cap W_k = W_j^{a_j \wedge a_k}$. However,

$$ \begin{aligned} \frac{h_{ijk}(v)}{h_{ijk}(u)} = 1 &\Rightarrow \left(\frac{v_i}{u_i}\right)^{q\varepsilon_{ijk}} \left(\frac{v_j}{u_j}\right)^{r\varepsilon_{jik}} \left(\frac{v_k}{u_k}\right)^{q'\varepsilon_{ijk}+r'\varepsilon_{jik}} = 1 \\ &\Rightarrow c_i^{q\varepsilon_{ijk}} c_j^{r\varepsilon_{jik}} c_k^{q'\varepsilon_{ijk}+r'\varepsilon_{jik}} = 1 \\ &\Rightarrow \left(c_i^q c_k^{q'}\right)^{\frac{(a_i \wedge a_k) \vee (a_j \wedge a_k)}{a_i \wedge a_k}} = \left(c_j^{-r} c_k^{-r'}\right)^{\frac{(a_i \wedge a_k) \vee (a_j \wedge a_k)}{a_j \wedge a_k}} \end{aligned} $$

Then, Lemma 3.3 claims that $W_i \cap W_k \cap W_j \cap W_k \ne \emptyset$. This implies $W_i \cap W_j \ne \emptyset$. Similar computations hold for $-n^- \le i < j \le 0 < k \le n^+$. That is, any pair in $\{W_i, -n^- \le i \le n^+\}$ has a non empty intersection. According to lemma 3.4, one then find some $z \in \bigcap_{i=-n^-}^{n^+} W_i$. This non zero complex number verifies $\rho(g_z^+)(u) = v$. $\square$

The separating set of Theorem 4.4 is not minimal in general. We provide a criterion to remove the redundant polynomials:

**Proposition 4.5.** The set $\mathcal{F}$ obtained removing $h_{ijk}$ whenever $a_k$ divides $(a_i \vee a_j)$ is a minimal separating set for the inclusion.

*Proof.* We try to remove a polynomial and determine if there exists two non separated orbits.

* Assume that we remove $p_{ij}$ for some $-n^- \le i < 0 \le j \le n^+$. We choose then $u_i = u_j = 1$ and $v_i = v_j = 2$ (other coordinates are supposed to be zero). Then, for any polynomial $P \in \mathcal{F}$, $P(u) = P(v) = 0$. However, for any $z \in \mathbb{C}^*$, $\rho(g_z^+)(u) = v \Rightarrow \left\{ \begin{array}{l} z^{a_i} = c_i = 2 \\ z^{a_j} = c_j = 2 \end{array} \right.$. This is impossible since $a_i$ and $a_j$ have opposite sign.

* Assume that we remove $h_{ijk}$ for some $-n^- \le k \le 0 \le i < j \le n^+$. If $a_k$ divides $(a_i \vee a_j)$ for some $-n^- \le i < j \le 0 < k \le n^+$, recall that

$$ \left\{ \begin{array}{l} p_{ik}(u) = p_{jk}(v) \\ p_{jk}(u) = p_{jk}(v) \end{array} \right. \Leftrightarrow \begin{array}{l} W_i \cap W_k \ne \emptyset \\ W_j \cap W_k \ne \emptyset \end{array} $$

But in that case, Lemma 3.6 ensures directly that

$$ \left\{ \begin{array}{l} W_i \cap W_k \neq \emptyset \\ W_j \cap W_k \neq \emptyset \end{array} \right. \Leftrightarrow W_i \cap W_j \cap W_k \neq \emptyset $$

Then, $u$ and $v$ are in the same orbit. On the opposite, suppose that $a_k$ does not divide $a_i \wedge a_j$. We choose $u_i = v_i = u_k = v_k = u_j = 1$ but $v_j = \exp\left(\frac{i a_j \pi}{a_k}\right)$. Then, $c_i = c_k = 1$ while $c_j = \exp\left(\frac{i a_j \pi}{a_k}\right)$. Since $W_i \cap W_k$ and $W_j \cap W_k$ are non empty, $p_{ik}(u) = p_{ik}(v)$ and $p_{jk}(u) = p_{jk}(v)$. All the other polynomials in $f$ vanish at $u$ and $v$. However, as discussed in the proof of lemma 3.6, $W_i \cap W_j = \emptyset$ and $u$ and $v$ are not in the same orbit. $\square$

**Remark 4.6.** The cardinal of $f$ is bounded by $n^+(n^- + 1) \le \#f \le n^+(n^- + 1) + (n^- + 1)(\binom{n^+}{2} + n^+\binom{n^-+1}{2})$. The lower bound is reached when all the integers $a_i$ are the same up to sign. The upper bound is reached when the integers $a_i$ are pairwise relatively prime.

Our separating set is much smaller than a generating set. The set of irreducible elements of $M_A^+$ (elements which are not the sum of two non trivial elements) corresponds to a minimal generating set. Algorithm 1.4.5 in [Stu08] computes them. Since irreducible elements come with many supports in $\llbracket n^-, n^+ \rrbracket$, their cardinality is exponential in the dimension. Whereas, the separating set of Theorem 4.4 consists of irreducible solutions of support of size at most three. It selects furthermore at most one monomial for each support. Its cardinality is thus less than $(\binom{N}{3}) \sim N^3$. The difference appears clearly in the following example:
**Example 4.7.** Let $(\rho, \hat{\mathcal{V}})$ be the representation of $\text{SO}_2(\mathbb{C})$ on $\mathcal{L}_{-11} \oplus \mathcal{L}_2 \oplus \mathcal{L}_3 \oplus \mathcal{L}_5 \oplus \mathcal{L}_7 \oplus \mathcal{L}_{11}$. A minimal generating set consists of the following 26 irreducible monomials:

$$ g := \left\{ \begin{array}{cccccccccc} v_{-1}v_5 & v_{-1}v_2v_3 & v_{-1}v_1v_2^3 & v_{-1}v_1v_2^2 & v_{-1}v_1v_4^2 & v_{-1}v_1v_3^3 & v_{-1}v_1v_4^4 & v_{-1}v_1v_3^3v_4 & v_{-1}v_1v_2v_3v_4^2 \\ v_{-1}v_2v_4^5 & v_{-1}v_1v_4^5 & v_{-1}v_1v_2v_3^2v_4 & v_{-1}v_1v_2v_4^2 & v_{-1}v_1v_2v_3^2v_4 & v_{-1}v_1v_2v_3^3 & v_{-1}v_1v_4^3 & v_{-1}v_1v_2v_3^6 & v_{-1}v_1v_2v_3^7 \\ v_{-1}v_2v_4^4 & v_{-1}v_1v_2^4 & v_{-1}v_1v_2v_3^3 & v_{-1}v_1v_2v_4^2 & v_{-1}v_1v_2v_3^2 & v_{-1}v_1v_2v_3^5 & v_{-1}v_1v_4^5 & v_{-1}v_1v_2v_3^6 & v_{-1}v_1v_2v_4^7 \\ v_{-1}v_2v_4^6 & v_{-1}v_1v_4^6 & v_{-1}v_1v_2v_3^2 & v_{-1}v_1v_2v_4^2 & v_{-1}v_1v_2v_3^2 & v_{-1}v_1v_2v_3^5 & v_{-1}v_1v_4^5 & v_{-1}v_1v_2v_3^6 & v_{-1}v_1v_2v_4^7 \end{array} \right\} $$

Whereas, Theorem 4.4 gives the following polynomial separating set of cardinal 11 (where 4 among the 10 polynomials of type $h_{ijk}$ have been removed):

$$ f := \left\{ \begin{array}{c} v_{-1}^2 v_1^{11} \quad v_1^3 v_2^{11} \quad v_2^5 v_3^{11} \quad v_3^7 v_4^{11} \quad v_4^7 v_5^{11} \quad v_5^7 v_6^{11} \\ h_{1,2,-1} = v_1^6 v_2^7 v_3^3 \quad h_{1,3,-1} = v_1^6 v_3^2 v_4^2 \quad h_{1,4,-1} = v_1^6 v_3^3 v_4^3 \\ h_{2,3,-1} = v_2^4 v_3^2 v_4^2 \quad h_{2,4,-1} = v_3^3 v_2^4 v_4^3 \quad h_{3,4,-1} = v_3^9 v_4^3 v_5^6 \end{array} \right\} $$

# 5 Separating invariants for $\text{SO}_2(\mathbb{R})$

Consider the action of $\text{SO}_2(\mathbb{R})$ on $\mathcal{V} = \bigoplus_{i=1}^n \mathcal{V}_{a_i}$ where $a_1, \dots, a_n \in \mathbb{N} \setminus \{0\}$. In this Section, we give a minimal separating set of cardinality $n^2$ (Theorem 5.2). Since the definition of separation over $\mathbb{R}$ is weaker, we obtain a lower cardinality than its complex analogue provided by Theorem 4.4, where the cardinality is dominated by $n^3$. Example 5.4 exhibits clearly this difference.

Since $\text{SO}_2(\mathbb{R})$ and $\text{O}_2(\mathbb{R})$ are both compact, so do their orbits and polynomials can separate all the orbit space. This gives rise to a more natural definition of separation:

**Definition 5.1.** Let $\mathcal{G}$ be a real compact algebraic group acting on a real variety $\mathcal{V}$. A subset $S \subset \mathbb{R}[\mathcal{V}]^G$ is a separating set if for any $u, v \in \mathcal{V}$, polynomials in $S$ agree on $u$ and $v$ iff $u$ and $v$ are in the same orbit.

Separating set over $\mathbb{R}$ are thus frequently smaller than over $\mathbb{C}$ [Duf09]. This difference matters since a great part of the applications concern only the real numbers.

The complexification $\hat{\mathcal{V}}$ decomposes into $\bigoplus_{i=1}^n (\mathcal{L}_{a_i} \oplus \mathcal{L}_{-a_i})$. We endow $\hat{\mathcal{V}}$ with the complex coordinates $v_1, v_{-1}, \dots, v_n, v_{-n}$ as in Section 4. This is, for any $v \in \mathcal{V}$ and $1 \le i \le n$, $v_i = \bar{v}_{-i}$. Note that we have $v_i \ne 0 \Leftrightarrow v_{-i} \ne 0$. Hence we can remove the difficult polynomials of type $h_{ijk}$, introduced for $\text{SO}_2(\mathbb{C})$.

**Theorem 5.2.** The following set of cardinality $n^2$ is a minimal separating set for $\text{SO}_2(\mathbb{R}) \cap \mathcal{V}$:

$$ f_{\mathbb{R}} = \left\{ \begin{array}{ll} 2\Re(p_{-ij}) = p_{-ij} + p_{-ji}, & 1 \le i \le j \le n \\ -2\Im(p_{-ij}) = i(p_{-ij} - p_{-ji}), & 1 \le i < j \le n \end{array} \right\} $$

*Proof.* Let $u$ and $v$ be two points in $\mathcal{V}$ such that for any $1 \le i \le j \le n$, $p_{-ij}(u) = p_{-ij}(v)$. For any integer $1 \le i \le n$, we note that $u_i = 0 \Leftrightarrow u_{-i} = 0 \Leftrightarrow \Re(p_{-ii}) = 0$. It remains then to separate the orbits in the sublinear space $\bigoplus_{j \ne i}^n \mathcal{V}_{a_j}$.

Hence we assume that none of the coordinates of $u$ and $v$ vanish. Then, quotients $c_i$ and wheels $W_i$ are well defined. For $1 \le i \le n$, $p_{ii}(u) = p_{ii}(v) \Rightarrow u_i u_{-i} = v_i v_{-i} \Rightarrow c_i = \frac{u_i}{v_i} = \frac{v_{-i}}{u_{-i}} = \frac{1}{c_i}$. Then, $|c_i| = 1$ and the wheel $W_i$ is included in the unit circle $\mathcal{U}$. As discussed in section 2.3, $z \in W_i \subset \mathcal{U} \Rightarrow g_z^+ \in \text{SO}_2(\mathbb{R})$. Furthermore as in Section 4, for $1 \le i < j \le n$, $p_{-ij}(u) = p_{-ij}(v) \Rightarrow W_i \cap W_j \ne \emptyset$. Then lemma 3.4 ensures that $\bigcap_{i=1}^n W_i$ is not empty. For some $z \in \bigcap_{i=1}^n W_i$, one has $g_z^+ \in \text{SO}_2(\mathbb{R})$ and $\rho(g_z^+)(v) = u$.

Unfortunately the restrictions on $\mathcal{V}$ of polynomials $p_{-ij} \in \mathbb{C}[\hat{\mathcal{V}}]^{\text{SO}_2(\mathbb{C})}$ are not real. We nevertheless notice that on $\mathcal{V}$, $p_{-ij} = \bar{p}_{-ji}$. Then, $p_{ij} = p_{-ij} + p_{-ji} = 2\Re(p_{-ij})$ and $i(p_{-ij} - p_{-ji}) = -2\Im(p_{-ij})$ are real. $\square$
**Remark 5.3.** When $a_1 = 1$ for any $1 \le i \le n$, the representation of $SO_2(\mathbb{R})$ on $\mathcal{V}$ is isomorphic to the representation on $2 \times n$ matrices $M_{2,n}(\mathbb{R})$. Then, $\mathfrak{R}(p_{-ij})$ is the standard scalar product between the $i^{th}$ and $j^{th}$ column, while $\mathfrak{S}(p_{-ij})$ is their determinant. Thus, the separating set $f_{\mathbb{R}}$ consists of the classical Weyl invariants [Wey46, Theorem 2.9.A].

**Example 5.4.** Consider the representation of $SO_2(\mathbb{R})$ on $\mathcal{V} = \mathcal{V}_2 \oplus \mathcal{V}_3 \oplus \mathcal{V}_5 \oplus \mathcal{V}_7$. Then, Theorem 5.2 provides a set of 16 invariant polynomials separating the orbits of $SO_2(\mathbb{R})$:

$$f_{\mathbb{R}} = \left\{ \begin{array}{c} \begin{array}{ccccccc} 2v_1v_{-1} & v_{-2}^2v_1^3 + v_{-1}^3v_2^2 & v_{-3}^2v_1^5 + v_{-1}^5v_3^2 & v_{-4}^2v_1^7 + v_{-1}^7v_4^2 & 2v_2v_{-2} \\ v_{-3}^3v_2^5 + v_{-2}^5v_3^3 & v_{-4}^3v_2^7 + v_{-2}^7v_4^3 & 2v_3v_{-3} & v_{-4}^5v_3^7 + v_{-3}^7v_4^5 & 2v_4v_{-4} \end{array} \\ \begin{array}{c} i(v_{-2}^2v_1^3 - v_{-1}^3v_2^2) \\ i(v_{-3}^3v_2^5 - v_{-2}^5v_3^2) \end{array} \begin{array}{c} i(v_{-3}^2v_1^5 - v_{-1}^5v_3^2) \\ i(v_{-4}^3v_2^7 - v_{-2}^7v_4^3) \end{array} \begin{array}{c} i(v_{-4}^2v_1^7 - v_{-1}^7v_4^2) \\ i(v_{-4}^5v_3^7 - v_{-3}^7v_4^5) \end{array} \end{array} \right\}$$

Whereas, 24 monomials among the 48 of type $h_{ijk}$ are required to separate orbits of $SO_2(\mathbb{C})$ in $\hat{\mathcal{V}}$. Theorem 4.4 provides thus a separating set of cardinality $16 + 24 = 40$ to separate orbits of the complex action.

# 6 Separating invariants for $O_2(\mathbb{R})$

We now consider actions of $O_2(\mathbb{R}) \cong SO_2(\mathbb{R}) \times C_2$. The twist induces the existence of second wheels $W_i^-$ corresponding to elements in $O_2(\mathbb{R})^-$ mapping $u_i$ to $v_{-i}$. Distinguishing positive and negative wheel requires to add invariants of support three or four. Thus the separating set has a cardinality in $O(N^4)$. Consider the representation of $O_2(\mathbb{R})$ on

$$\mathcal{V} := \bigoplus_{i=1}^{m} \mathcal{V}_{-1} \oplus \bigoplus_{i=1}^{n} \mathcal{V}_{a_i}$$

endowed with the coordinates $(d_1, ..., d_m, v_1, v_{-1}, ..., v_n, v_{-n})$. We compute a separating set in Theorem 6.5, which is not minimal in general for the inclusion. We next give a criterion to remove redundant polynomials and get a minimal separating set in Proposition 6.7. We finish comparing the obtained separating set to a minimal generating set provided by an algorithm in [Des+23].

## 6.1 Computation of the set

We prove that the following set separates real orbits

$$\mathcal{F}_{\mathbb{R}} = \left\{ \begin{array}{l} P_{ij} = p_{-ij} + p_{-ji} \qquad 1 \le i \le j \le n \\ T_{ijk} = (p_{-ik} - p_{-ki})(p_{-jk} - p_{-kj}), \quad (i, j, k) \in \mathcal{T}(n) \\ Q_{ijkl} = (p_{-ij} - p_{-ji})(p_{-kl} - p_{-lk}), \quad i, j, k, l \in \mathcal{Q}(n) \\ D_{st} = d_s d_t, \qquad 1 \le s \le t \le m \\ S_{ijs} = i(p_{-ij} - p_{-ji}) d_s, \qquad 1 \le i < j \le n, \quad 1 \le s \le m \end{array} \right\}$$

where $\mathcal{T}(n) \subset [1, n]^3$ and $\mathcal{Q}(n) \subset [1, n]^4$ are the sets of ordered triplets and quadruplets:

$$(i, j, k) \in \mathcal{T}(n) \Leftrightarrow \left\{ \begin{array}{l} i < j, k < l \\ k \in [1, n] \setminus \{i, j\} \end{array} \right. \quad (i, j, k, l) \in \mathcal{Q}(n) \Leftrightarrow \left\{ \begin{array}{l} i < j, k < l \\ \{i, j\} \cap \{k, l\} = \emptyset \\ i + j \le k + l \\ \text{If } i + j = k + l, i < k \end{array} \right.$$

Consider $u = (d_1, ..., d_q, u_1, u_{-1}, ..., u_n, u_{-n})$ and $v = (d'_1, ..., d'_q, v_1, v_{-1}, ..., v_n, v_{-n})$ two vectors where $\mathcal{F}_{\mathbb{R}}$ agree. In addition to the wheels $W_i$ and $W_{-i}$ introduced in Section 4, we note $W_i^-$ and $W_{-i}^-$ the set of complex numbers such that

$$\left\{ \begin{array}{ll} \forall z \in W_i^-, & \rho_{a_i}(g_z^-)(u_i, u_{-i}) = (*, v_{-i}) \\ \forall z \in W_{-i}^-, & \rho_{a_i}(g_z^-)(u_i, u_{-i}) = (v_i, *) \end{array} \right.$$
As for Theorem 5.2 we assume that none of the coordinates of u and v vanish. Thus the quotients $c_i^- = \frac{v_i}{u_i}$ and $c_{-i}^- = \frac{v_i}{u_{-i}}$ are well defined and we have $W_i^- = W_{-i}^- = W_{-i}^{a_i}$ and $W_{-i}^- = W_{-i}^{-a_i}$. For any $1 \le i \le n$, we note $W_I = W_i \cap W_{-i}$ (the set of complex numbers such that $\rho_{a_i}(g_z^+)(u_i, u_{-i}) = (v_i, v_{-i})$ and $W_I^- = W_i^- \cap W_{-i}^-$ (the set of complex numbers such that $\rho_{a_i}(g_z^-)(u_i, u_{-i}) = (v_i, v_{-i})$). Furthermore, for a list $i_1, ..., i_n$ of indices, we note $W_{i_1...i_n} = W_{i_1} \cap ... \cap W_{i_n}$.

$$ P_{ii}(u) = P_{ii}(v) \Rightarrow u_i u_{-i} = v_i v_{-i} \Rightarrow \begin{cases} c_i = \frac{v_i}{u_i} = \frac{u_{-i}}{v_{-i}} = \frac{1}{c_i} \\ c_i^- = \frac{v_{-i}}{u_i} = \frac{v_i}{v_{-i}} = \frac{1}{c_i^-} \end{cases} $$

Then, $|c_i| = |c_i^-| = 1$ and the wheels $W_i, W_i^-$ are both included in the unit circle $\mathcal{U}$. As discussed in Section 2.3, $z \in \mathcal{U} \Rightarrow g_z^+, g_z^- \in O_2(\mathbb{R})$. We next prove the following sequence of lemmas:

**Lemma 6.1.** For any integer $1 \le i \le n$, we have $W_i = W_{-i} = W_I$ and $W_i^- = W_{-i}^- = W_I^-$.

*Proof.* By assumption we have $P_{ii}(u) = P_{ii}(v) \Rightarrow u_i u_{-i} = v_i v_{-i}$. Since no coordinate vanish, $c_i = \frac{v_i}{u_i} = \frac{u_{-i}}{v_{-i}} = \frac{1}{c_i^-}$. This implies $W_i = W_{-i}$ and $W_i^- = W_{-i}^-$.

**Lemma 6.2.** For any integers $1 \le i < j \le n$, we have $W_{IJ} \ne \emptyset$ or $W_{IJ}^- \ne \emptyset$.

*Proof.* We notice that $p_{-ij}(v) = v_{-i}^{\varepsilon_{ij}} v_j^{\varepsilon_{ji}} = u_{-i}^{\varepsilon_{ij}} c_i^{\varepsilon_{ij}} u_j^{\varepsilon_{ji}} c_j^{\varepsilon_{ji}} = c_i^{\varepsilon_{ij}} c_j^{\varepsilon_{ji}} p_{-ij}(u)$. It provides

$$ \begin{aligned} P_{-ij}(u) = P_{-ij}(v) &\Leftrightarrow p_{-ij}(u) + p_{-ji}(u) = p_{-ij}(v) + p_{-ji}(v) \\ &\Leftrightarrow p_{-ij}(u) + p_{-ji}(u) = c_i^{\varepsilon_{ij}} c_j^{\varepsilon_{ji}} p_{-ij}(u) + c_{-i}^{\varepsilon_{ij}} c_{-j}^{\varepsilon_{ji}} p_{-ji}(u) \\ &\Leftrightarrow p_{-ij}(u) + p_{-ji}(u) = c_i^{\varepsilon_{ij}} c_j^{\varepsilon_{ji}} p_{-ij}(u) + c_i^{-\varepsilon_{ij}} c_j^{-\varepsilon_{ji}} p_{-ji}(u) \end{aligned} $$

This can be viewed as a second order equation of unknown $c_i^{\varepsilon_{ij}} c_j^{\varepsilon_{ji}}$, which has the two solutions $c_i^{\varepsilon_{ij}} c_j^{\varepsilon_{ji}} = 1$ or $c_i^{\varepsilon_{ij}} c_j^{\varepsilon_{ji}} = \frac{p_{-ji}(u)}{p_{-ij}(u)}$. The first solution implies $W_{ij} \ne \emptyset$ by lemma 3.3, and lemma 6.1 completes in $W_{IJ} \ne \emptyset$. The second solution gives $p_{-ij}(u) = c_i^{-\varepsilon_{ij}} c_j^{-\varepsilon_{ji}} p_{-ji}(u) = p_{-ji}(v)$, and

$$ p_{-ij}(v) = p_{-ji}(u) \Rightarrow v_{-i}^{\varepsilon_{ij}} v_j^{\varepsilon_{ji}} = u_{-j}^{\varepsilon_{ji}} u_i^{\varepsilon_{ij}} \Rightarrow \left(\frac{v_i}{u_i}\right)^{\varepsilon_{ij}} = \left(\frac{v_j}{u_{-j}}\right)^{\varepsilon_{ji}} \Rightarrow (c_i^-)^{\varepsilon_{ij}} = (c_{-j}^-)^{\varepsilon_{ji}} $$

Then lemma 3.3 claims that $W_{-ij}^- \ne \emptyset$, and lemma 6.1 completes in $W_{IJ}^- \ne \emptyset$.

**Lemma 6.3.** Let $(i, j, k) \in \mathcal{T}(n)$. Then, one has $\begin{cases} W_{IK} \ne \emptyset \\ W_{JK} \ne \emptyset \end{cases}$ or $\begin{cases} W_{IK}^- \ne \emptyset \\ W_{JK}^- \ne \emptyset \end{cases}$.

*Proof.* According to Lemma 6.2 we have $p_{-ij}(u) = p_{-ij}(v)$ (case $\delta_{ij} = +$) or $p_{-ij}(u) = p_{-ji}(v)$ (case $\delta_{ij} = -$). This provides $(p_{-ij} - p_{-ji})(u) = \delta_{ij}(p_{-ij} - p_{-ji})(v)$ and Lemma 6.2 adds that $W_{IJ}^{\delta_{ij}} \ne \emptyset$. This holds also for $j, k$ with a coefficient $\delta_{jk} = \pm$. However we have $T_{ijk}(u) = T_{ijk}(v)$:

$$ (p_{-ik} - p_{-ki})(-p_{-jk} - p_{-kj})(u) = (p_{-ik} - p_{-ki})(-p_{-jk} - p_{-kj})(v) $$

* If $T_{ijk}(u) \ne 0$, it gives $\begin{cases} (p_{-ik} - p_{-ki})(u) = \delta(p_{-ik} - p_{-ki})(v) \ne 0 \\ (p_{-jk} - p_{-kj})(u) = \delta(p_{-jk} - p_{-kj})(v) \ne 0 \end{cases}$ for a common $\delta = \delta_{ij} = \delta_{jk}$. Then, we have $\begin{cases} W_{IK}^\delta \ne \emptyset \\ W_{JK}^\delta \ne \emptyset \end{cases}$.

* If $T_{ijk}(u) = 0$, suppose for instance that $(p_{-ik} - p_{-ki})(u) = 0$. Then $p_{-ik}(u) = p_{-ik}(v) = p_{-ki}(u) = p_{-ki}(v) \ne 0$. We thus have both $W_{IK} \ne \emptyset$ and $W_{IK}^- \ne \emptyset$. However Lemma 6.2 ensures that $W_{KJ} \ne \emptyset$ or $W_{KJ}^- \ne \emptyset$. We deduce that $\begin{cases} W_{IJ} \ne \emptyset \\ W_{KJ} \ne \emptyset \end{cases}$ or $\begin{cases} W_{IJ}^- \ne \emptyset \\ W_{KJ}^- \ne \emptyset \end{cases}$.
**Lemma 6.4.** Let $(i, j, k, l) \in \mathcal{Q}(n)$. Then one has
$$ \begin{cases} W_{IJ} \neq \emptyset & \text{or} \quad \begin{cases} W_{IJ}^{-} \neq \emptyset \\ W_{KL}^{-} \neq \emptyset \end{cases} \\ W_{KL} \neq \emptyset & \end{cases} $$

*Proof.* It is the same proof than for Lemma 6.3, but with the polynomial equality $Q_{ijkl}(u) = Q_{ijkl}(v)$. $\square$

We are now able to establish the Theorem 6.5:

**Theorem 6.5.** The set of polynomials $\mathcal{F}_{\mathbb{R}}$ separates orbits with respect to the action $O_2(\mathbb{R}) \backsim \mathcal{V}$.

*Proof.* To prove it, we take $u$ and $v$ two vectors in $\mathcal{V}$ where polynomials of $\mathcal{F}_{\mathbb{R}}$ agree and exhibit an element $g \in O_2(\mathbb{R})$ mapping $u$ to $v$.

According to Lemmas 6.3 and 6.4 we have

$$ \forall 1 \le i < j \le n \text{ and } 1 \le k < l \le n, \quad \begin{cases} W_{IJ} \ne \emptyset \\ W_{KL} \ne \emptyset \end{cases} \text{ or } \begin{cases} W_{IJ}^{-} \ne \emptyset \\ W_{KL}^{-} \ne \emptyset \end{cases} $$

This is equivalent to $(\forall 1 \le i < j \le n, W_{IJ} \ne \emptyset)$ or $(\forall 1 \le i < j \le n, W_{IJ}^{-} \ne \emptyset)$. Then, according to Corollary 3.4, there exists $z \in \bigcap_{i=1}^{n} W_{I}^{+}$ or $z \in \bigcap_{i=1}^{n} W_{I}^{-}$.

Furthermore for any $1 \le s \le m$, the polynomial $D_{ss}$ ensures that $|d_s| = |d'_s|$. This is, $\rho_{-1}(g_z^+)(d_s) = d'_s$ or $\rho_{-1}(g_z^-)(d_s) = d'_s$. If one has $(+)$ but not $(-)$ then there is $1 \le i < j \le n$ with $W_{IJ}^{-} = \emptyset$, and this is $(p_{-ij} - p_{-ji})(u) = (p_{-ij} - p_{-ji})(v) \ne 0$. One notices that for any $1 \le s \le m$, $S_{ijs}(u) = S_{ijs}(v) \Rightarrow d_s = d'_s$. Then, $\rho(g_z^+)(u) = v$. Conversely, if one has $(+)$ but not $(-)$ then there is $1 \le i < j \le n$ with $W_{IJ}^{-} = \emptyset$, and this is $(p_{-ij} - p_{-ji})(u) = -(p_{-ij} - p_{-ji})(v) \ne 0$. One notices that for any $1 \le s \le m$, $S_{ijs}(u) = S_{ijs}(v) \Rightarrow d_s = -d'_s$. Then, $\rho(g_z^-)(u) = v$.

Suppose now that we have both $(+)$ and $(-)$. Note that for any $1 \le s \le t \le m$ with $d_s \ne 0$ and $d_t \ne 0$, $D_{st}(u) = D_{st}(v) \Rightarrow \frac{d_s}{d'_s} = \frac{d'_t}{d'_t}$. Suppose that all these quotients are positive. Then $\rho(g_z^+)(u) = v$. Conversely, if all these quotients are negative, then $\rho(g_z^-)(u) = v$. Anyway there exists some $g \in O_2(\mathbb{R})$ verifying $\rho(g)(u) = v$ and Theorem 6.5 is proven. $\square$

In some circumstances, one assumes that one of the coordinate does not vanish. It is the case, for instance, when we identify a Seshadri slice in a representation of $SO_3(\mathbb{R})$ on $\mathbb{R}^3 \oplus \mathcal{V}$ [HJ24]. Then Seshadri slice Lemma restricts to invariants of $O_2(\mathbb{R})$ on an open subset $\mathcal{Z} \subset \mathcal{V}_{-1} \oplus \mathcal{V}$ defined by $\{v, d_1 \ne 0\}$. The separating set is drastically reduced:

**Corollary 6.6.** Consider the action of $O_2(\mathbb{R})$ on $\mathcal{V}^* := \mathcal{V}_{-1}^* \oplus \bigoplus_{i=2}^{m} \mathcal{V}_{-1} \oplus \bigoplus_{i=1}^{n} \mathcal{V}_{a_i}$, where $\mathcal{V}^* = \mathcal{V}_{-1} \setminus \{0\}$. Then we have the following separating set of cardinality $m + n^2$:

$$ \mathcal{F}_{\mathbb{R}}^* = \left\{ \begin{array}{ll} P_{ij} = \frac{1}{2}(p_{-ij} + p_{-ji}) & 1 \le i \le j \le n \\ D_{1s} = d_1 d_s, & 1 \le s \le m \\ S_{ij1} = i(p_{-ij} - p_{-ji}) d_1, & 1 \le i < j \le n \end{array} \right\} $$

*Proof.* Consider $u = (d_1, ..., d_q, u_1, u_{-1}, ..., u_n, u_{-n})$ and $v = (d'_1, ..., d'_q, v_1, v_{-1}, ..., v_n, v_{-n})$ two points in $\mathcal{V}^*$ where polynomials of $\mathcal{F}_{\mathbb{R}}^*$ agree. Since $d'_1 \ne 0$ we can fix $\delta = \frac{d'_1}{d'_1} = \pm 1$. Then, $\forall 2 \le s \le m$, $D_{1s}(u) = D_{1s}(v) \Rightarrow \frac{d_s}{d'_s} = \delta$.

Furthermore, for all $1 \le i \le j \le n$, $P_{ij}(u) = P_{ij}(v)$ ensures that $W_{IJ} \ne \emptyset$ or $W_{IJ}^{-} \ne \emptyset$. Then, $S_{ij1}(u) = S_{ij1}(v) \Rightarrow W_{IJ}^{\delta} \ne \emptyset$. Corollary 3.5 claims that there exists some $z \in \bigcap_{i=1}^{n} W_{IJ}^{\delta}$. This complex verifies $\rho(g_z^{\delta})(u) = v$. $\square$
## 6.2 Minimality

In general, the separating set of Theorem 6.5 is not minimal for inclusion. We identify the redundant polynomials:

**Proposition 6.7.** Upon the following removals, $\mathcal{F}_R$ is a minimal separating set:

* For $(i, j, k) \in \mathcal{T}(n)$, if $a_k$ divides $a_i \lor a_j$, we keep $T_{ijk}$ but remove $T_{jki}$ and $T_{ikj}$.

* If furthermore $a_i$ divides $(a_k \lor a_j)$ and $a_j$ divides $(a_i \lor a_k)$, we remove $T_{ijk}$.

* For $(i, j, k, l) \in \mathcal{Q}(n)$, if $a_i \land a_j$ divides $a_k \lor a_l$ or $a_k \land a_l$ divides $a_i \lor a_j$, we remove $Q_{ijkl}$.

*Proof.* Again we try to remove a polynomial and determine if there exists two non separated orbits. Recall that we need points in the real trace, that is points verifying $v_i = \bar{v}_i$ for all $i$. For the polynomials of type $P$ we refer to section 4.

* Assume that we remove $T_{jki}$ and $T_{ikj}$ for some $(i, j, k) \in \mathcal{T}(n)$. By lemma 6.3, $T_{ijk}(u) = T_{ijk}(v)$ ensures that $\begin{cases} W_{IK}^{\delta} \neq \emptyset \\ W_{JK}^{\delta} \neq \emptyset \end{cases}$ for some $\delta = \pm 1$. Then, if $a_k$ divides $a_i \lor a_j$, it results directly by Proposition 3.5 that $W_{IJK}^{\delta} \neq \emptyset$. This is $u$ and $v$ belong to the same orbit.

On the opposite, suppose that we remove the polynomial $T_{ijk}$ although $a_i$ does not divide $a_k \lor a_j$ and $a_j$ does not divide $a_k \lor a_i$. We choose $u$ and $v$ as in the following table.

<table><thead><tr><th></th><th>i</th><th>-i</th><th>j</th><th>-j</th><th>k</th><th>-k</th></tr></thead><tbody><tr><th>u</th><td>1</td><td>1</td><td>1</td><td>1</td><td>exp(-ia<sub>k</sub>π(1/a<sub>i</sub> + 1/a<sub>j</sub>))</td><td>exp(ia<sub>k</sub>π(1/a<sub>i</sub> + 1/a<sub>j</sub>))</td></tr><tr><th>v</th><td>1</td><td>1</td><td>1</td><td>1</td><td>exp(ia<sub>k</sub>π(1/a<sub>i</sub> - 1/a<sub>j</sub>))</td><td>exp(ia<sub>k</sub>π(1/a<sub>j</sub> - 1/a<sub>i</sub>))</td></tr></tbody></table>

These choices clearly provide $c_i = c_i^- = c_j = c_j^- = 1$. We also compute $c_k = \exp(\frac{2i\pi a_k}{a_i})$ and $c_k^- = \exp(\frac{2i\pi a_k}{a_j})$. Note for instance that that $c_k^{\frac{a_j \lor a_k}{a_k}} c_j^{\frac{a_j \lor a_k}{a_j}} = \exp(\frac{2i\pi a_k}{a_i} \frac{a_j \lor a_k}{a_k}) = \exp(2i\pi \frac{a_j \lor a_k}{a_i}) \neq 1$. Lemma 3.3 provides by that way the intersections described in the right figure. We deduce that polynomials of type $P$ agree on $u$ and $v$. So do $T_{jki}$ and $T_{ikj}$. All the others polynomials in $\mathcal{F}_R$ vanish. Altough, $u$ and $v$ are not in the same orbit.

<figure><img src="image_3.png" /></figure>

* Assume that we remove $Q_{ijkl}$ for some $(i, j, k, l) \in \mathcal{Q}(n)$ such that $a_i \land a_j$ divides $a_k \lor a_l$. Take $u$ and $v$ such that $\begin{cases} W_{IJ}^{\delta} \neq \emptyset \\ W_{IJ}^{\delta} = \emptyset \end{cases}$ for some $\delta = \pm 1$. By Lemma 6.3, polynomials of type $T$ ensure that $W_{IJ}^{\delta} \cap W_{K}^{\delta} \neq \emptyset$ and $W_{IJ}^{\delta} \cap W_{L}^{\delta} \neq \emptyset$. However, the well $W_{IJ}^{\delta}$ is of cardinality $a_i \land a_j$. Then, the criterion of Proposition 3.6 gives $W_{KL}^{\delta} \neq \emptyset$, and $u$ and $v$ are in the same orbit.

On the opposite, assume that $a_i \land a_j$ does not divide $a_k \lor a_l$ and $a_k \land a_l$ does not divide $a_i \lor a_j$. We choose $u$ and $v$ as in the following table.

<table><thead><tr><th></th><th>i</th><th>-i</th><th>j</th><th>-j</th><th>k</th><th>-k</th><th>l</th><th>-l</th></tr></thead><tbody><tr><th>u</th><td>1</td><td>1</td><td>exp(-i(a<sub>j</sub>π/a<sub>k</sub>∧a<sub>l</sub>))</td><td>exp(i(a<sub>j</sub>π/a<sub>k</sub>∧a<sub>l</sub>))</td><td>1</td><td>1</td><td>exp(-i(a<sub>l</sub>π/a<sub>i</sub>∧a<sub>j</sub>))</td><td>exp(i(a<sub>l</sub>π/a<sub>i</sub>∧a<sub>j</sub>))</td></tr><tr><th>v</th><td>1</td><td>1</td><td>exp(-i(a<sub>j</sub>π/a<sub>k</sub>∧a<sub>l</sub>))</td><td>exp(i(a<sub>j</sub>π/a<sub>k</sub>∧a<sub>l</sub>))</td><td>1</td><td>1</td><td>exp(i(a<sub>l</sub>π/a<sub>i</sub>∧a<sub>j</sub>))</td><td>exp(-i(a<sub>l</sub>π/a<sub>i</sub>∧a<sub>j</sub>))</td></tr></tbody></table>

These choices clearly provide $c_i = c_i^- = c_j = c_j^- = 1$. We find also $c_j^- = \exp(\frac{2i\pi a_j}{a_k \land a_l})$ and $c_l = \exp(\frac{2i\pi a_l}{a_i \land a_j})$. Note for instance that $(c_i^-)^{\frac{a_i \lor a_j}{a_i}} (c_j^-)^{\frac{a_i \lor a_j}{a_j}} = \exp(\frac{2i\pi a_j}{a_i \land a_l} \frac{a_i \lor a_j}{a_j}) = \exp(\frac{2i\pi a_j}{a_k \land a_l}) \neq 1$. Lemma 3.3 provides by that way the intersections described in the right figure. We deduce that polynomials of type $P$ and $T$ agree on $u$ and $v$. So do $Q_{ikjl}$ and $Q_{iljk}$. All the others polynomials in $\mathcal{F}_R$ vanish. Altough, $u$ and $v$ are not in the same orbit.

<figure><img src="image_4.png" /></figure>
• Assume that we remove $D_{ss}$ for some $1 \le s \le m$. We choose then $d_s = 1$ but $d'_s = 2$. Then all the polynomials in $\mathcal{F}_R$ vanish. Altough, $u$ and $v$ are not in the same orbit.

• Assume that we remove $D_{st}$ for some $1 \le s < t \le m$. We choose then $d_s = d'_s = d_t = 1$ but $d'_t = -1$. Then $D_{ss}(u) = D_{ss}(v) = D_{tt}(u) = D_{tt}(v) = 1$ while all the others polynomials in $\mathcal{F}_R$ vanish. Altough, $u$ and $v$ are not in the same orbit.

• Assume that we remove $S_{ijs}$ for some $1 \le i < j \le n$ and $1 \le s \le q$. We choose then $d_s = 1$, $d'_s = -1$, $(u_i, u_{-i}) = (v_i, v_{-i}) = (1, 1)$ and $(u_j, u_{-j}) = (v_j, v_{-j}) = (\exp(i\pi\sqrt{2}), \exp(-i\pi\sqrt{2}))$. Here $W_{IJ}$ is not empty because it contains at least 1, while $W_{IJ}^{-}$ is empty because $\sqrt{2}$ is not rational. Then $P_{ii}$, $P_{ij}$ and $P_{jj}$ agree on $u$ and $v$. All the other polynomials in $\mathcal{F}_R$ vanish on $u$ and $v$. Altough, $W_{IJ}^{-} = \emptyset$, thus $u$ and $v$ are not in the same orbit. $\square$

**Remark 6.8.** The obtained minimal separating set thus has a cardinality bounded by

$$n^2 \sim \underbrace{\frac{n(n+1)}{2}}_{P_{ij}} + \underbrace{\frac{m(m+1)}{2}}_{D_{st}} + \underbrace{m\binom{n}{2}}_{S_{ijs}} \le \#\mathcal{F}_R \le \underbrace{\frac{n(n+1)}{2}}_{P_{ij}} + \underbrace{3\binom{n}{3}}_{T_{ijk}} + \underbrace{3\binom{n}{4}}_{Q_{ijkl}} + \underbrace{\frac{m(m+1)}{2}}_{D_{st}} + \underbrace{m\binom{n}{2}}_{S_{ijs}} \sim n^4$$

**Remark 6.9.** The lower bound is reached when all the integers $a_i$ are equal to 1. in that case, the representation of $O_2(\mathbb{R})$ on $\mathcal{V}$ is isomorphic to the representation on $2 \times n$ matrices $\mathcal{M}_{2,n}(\mathbb{R})$. While the polynomials of type T and Q are removed, the polynomials $P_{ij}$ are the scalar product between the $i^{th}$ and $j^{th}$ columns. Here again, the separating set $\mathcal{F}_R$ consists of the well known Weyl invariants [Wey46, Theorem 2.9.A].

## 6.3 Comparison to a generating set

Since the property to separate orbit is weaker than to generate the invariant algebra, separating sets can have drastically lower cardinalities than generating sets. However, they are sufficient for many applications, as designing invariant neural networks or stratifying the orbit space. In this Section, we bring to light this difference of cardinality, comparing the separating set of Theorem 6.5 with a generating set provided by [Des+23]. Consider the action of $O_2(\mathbb{R})$ on $\mathcal{V} = \mathcal{V}_{-1}^m \oplus \bigoplus_{i=1}^n \mathcal{V}_{a_i}$ with coordinates $(d_1, ..., d_m, v_1, v_{-1}, ..., v_n, v_{-n})$. We note $Z := \{v_iv_{-i}, 1 \le i \le n\}$ the set of irreducible invariant monomials that are real.

**Theorem 6.10.** [Des+23, Theorem 5.5] The set of polynomials

$$B = \{ D_{st} = d_s d_t, \quad R_i = y_i y_{-i}, \quad R_e = \Re(m_e), \quad I_{ef} = \Im(m_e) \Im(m_f), \quad S_{se} = d_s \Im(m_e) \}$$

where $m_e, m_f$ are chosen such that neither $m_e m_f$ nor $m_e \overline{m_f}$ contains a real monomial $v_i v_{-i} \in Z$, generates the invariant algebra $\mathbb{R}[\mathcal{V}]^{O_2(\mathbb{R})}$.

The set given by Theorem 6.10 is not minimal for inclusion. It can be significatively trimmed by a complementary algorithm explained in [Des+23, Section 6]. This algorithm relies on the fact that, thanks to Hilbert series, we know in advance the dimension of the linear space of invariants for a given multi degree. Most of polynomials among $I_{ef}$ can in fact be eliminated this way. Nevertheless, this task has heavy cost. As in Section 4 the generating set is composed of monomials of any support in $[1, n]$, and the cardinality grows like $2^n$. While, the separating set contains only monomials with support of size at most 4, thus the cardinality is dominated by $\binom{n}{4} \sim n^4$. Furthermore, in practice, most of the polynomials of type $Q_{ijkl}$ are removed (see Example 6.11). We have implemented the algorithm with Maple to make the effective comparison:

**Example 6.11.** Consider the action of $O_2(\mathbb{R})$ on $\mathcal{V} = \mathcal{V}_{-1}^3 \oplus \mathcal{V}_2 \oplus \mathcal{V}_3 \oplus \mathcal{V}_5 \oplus \mathcal{V}_7$. Algorithm 1.4.5 in [Stu08] provides a set of cardinality $\#\mathcal{M} = 64$ complex irreducible monomials which are invariant for the action of $SO_2(\mathbb{C})$. Among them are the four monomials $Z := \{v_1 v_{-1}, v_2 v_{-2}, v_3 v_{-3}, v_4 v_{-4}\}$ and 60 monomials
$\mathfrak{m}_e \in \mathcal{M} \setminus Z$ with $\mathfrak{m}_e \neq \overline{\mathfrak{m}_e}$. They induce 3 polynomials of type $I_{ef}$. We obtain then a generating set $\mathcal{G}$ of cardinality $6+4+3+3 \cdot 30+3=133$ for the action of $O_2(\mathbb{C})$:

$$ \mathcal{G} = \left\{ \begin{array}{c} \begin{array}{cccccc} d_1^2 & d_1d_2 & d_1d_3 & d_2^2 & d_2d_3 & d_3^2 \\ v_1v_{-1} & v_2v_{-2} & v_3v_{-3} & v_4v_{-4} & & \\ \mathfrak{m}_e + \overline{\mathfrak{m}_e}, & \mathfrak{m}_e \in \mathcal{M} \setminus Z & d_s(\mathfrak{m}_e - \overline{\mathfrak{m}_e}), & 1 \le s \le 3 & \text{and } \mathfrak{m}_e \in \mathcal{M} \setminus Z \end{array} \\ (-v_{-2}^2v_1^3 + v_{-1}^3v_2^2) (-v_{-4}^5v_3^7 + v_{-3}^7v_4^5) \quad (-v_{-3}^2v_1^5 + v_{-1}^5v_3^2) (-v_{-4}^3v_2^7 + v_{-2}^7v_4^3) \\ (-v_{-3}^3v_2^5 + v_{-2}^5v_3^3) (-v_{-4}^2v_1^7 + v_{-1}^7v_4^2) \end{array} \right\} $$

The elimination algorithm [Des+23, Section 6], although heavy, cannot remove more than the three last polynomials. Whereas, Theorem 6.5 provides a separating set of cardinality 46, where all the polynomials of type $Q_{ijkl}$ have been removed:

$$ \mathcal{F}_{\mathbb{R}} := \left\{ \begin{array}{c} \begin{array}{cccccc} d_1^2 & d_1d_2 & d_1d_3 & d_2^2 & d_2d_3 & d_3^2 \\ 2v_1v_{-1} & v_{-2}^2v_1^3 + v_{-1}^3v_2^2 & v_{-2}^2v_1^5 + v_{-1}^5v_2^2 & v_{-4}^2v_1^7 + v_{-1}^7v_2^4 & 2v_2v_{-2} & \\ v_{-3}^3v_2^5 + v_{-2}^5v_3^3 & v_{-4}^3v_2^7 + v_{-2}^7v_4^3 & 2v_3v_{-3} & v_{-4}^5v_3^7 + v_{-3}^7v_4^5 & 2v_4v_{-4} & \\ & T_{1,2,3} & T_{1,3,2} & T_{2,3,1} & T_{1,2,4} & T_{1,4,2} \\ & T_{1,3,4} & T_{1,4,3} & T_{3,4,1} & T_{2,3,4} & T_{2,4,3} \\ & & & & T_{3,4,2} & \\ d_1(v_{-2}^2v_1^3 - v_{-1}^3v_2^2) & i & d_2(v_{-2}^2v_1^3 - v_{-1}^3v_2^2) & i & d_3(v_{-2}^2v_1^3 - v_{-1}^3v_2^2) & i & d_1(v_{-3}^2v_1^5 - v_{-1}^5v_3^2) & i \\ d_2(v_{-2}^2v_1^5 - v_{-1}^5v_3^2) & i & d_3(v_{-2}^2v_1^5 - v_{-1}^5v_3^2) & i & d_1(v_{-2}^2v_1^7 - v_{-1}^7v_4^2) & i & d_2(v_{-2}^2v_1^7 - v_{-1}^7v_4^2) & i \\ d_3(v_{-4}^2v_1^7 - v_{-1}^7v_4^2) & i & d_1(v_{-3}^3v_2^5 - v_{-2}^5v_3^3) & i & d_2(v_{-3}^3v_2^5 - v_{-2}^5v_3^3) & i & d_3(v_{-3}^3v_2^5 - v_{-2}^5v_3^3) & i \\ d_1(v_{-4}^3v_2^7 - v_{-2}^7v_4^3) & i & d_2(v_{-4}^3v_2^7 - v_{-2}^7v_4^3) & i & d_3(v_{-4}^3v_2^7 - v_{-2}^7v_4^3) & i & d_1(v_{-4}^5v_3^7 - v_{-3}^7v_4^5) & i \\ d_2(v_{-5}^4v_3^7 - v_{-3}^7v_4^5) & i & d_3(v_{-5}^4v_3^7 - v_{-3}^7v_4^5) & i & & & & \end{array} \right\} $$

# 7 Separating invariants for $O_2(\mathbb{C})$

In this section, we consider the representation of $O_2(\mathbb{C})$ on $\hat{\mathcal{V}} = \bigoplus_{i=1}^m \hat{\mathcal{V}}_{-1} \oplus \bigoplus_{i=1}^n \hat{\mathcal{V}}_{a_i}$. where $a_1, ..., a_n \in \mathbb{N} \setminus \{0\}$.

We compute a polynomial separating set in (Theorem 7.4). As for $O_2(\mathbb{R})$, we delete redundant polynomials and get a minimal separating set in Proposition 7.5.

## 7.1 Computation of the set

We prove that the following set of invariant polynomials separate closed orbits:

$$ \mathcal{F} = \left\{ \begin{array}{l} P_{ij} = p_{-ij} + p_{-ji} \qquad 1 \le i \le j \le n \\ H_{ijk} = h_{ij-k} + h_{-i-jk} \qquad (i, j, k) \in \mathcal{T}(n) \\ T_{ijk} = (p_{-ik} - p_{-ki})(p_{-jk} - p_{-kj}), \qquad (i, j, k) \in \mathcal{T}(n) \\ Q_{ijkl} = (p_{-ij} - p_{-ji})(p_{-kl} - p_{-lk}), \qquad (i, j, k, l) \in \mathcal{Q}(n) \\ D_{st} = d_s d_t, \qquad 1 \le s \le t \le m \\ S_{ijs} = i(p_{-ij} - p_{-ji}) d_s, \qquad 1 \le i < j \le n \text{ and } 1 \le s \le m \end{array} \right\} $$

Consider two vectors $u = (d_1, ..., d_q, u_1, u_{-1}, ..., u_n, u_{-n})$ and $v = (d'_1, ..., d'_q, v_1, v_{-1}, ..., v_n, v_{-n})$ in closed orbits such that for any $P \in \mathcal{F}$, $P(u) = P(v)$. When none of the coordinates vanish, the sequence of Lemmas in Section 6 still holds with the same proofs. Hence, the real separating set $\mathcal{F}_{\mathbb{R}} \subset \mathcal{F}$ is sufficient to separate their orbits.

The great difficulty over $\mathbb{C}$ is the vanishing coordinates: it is possible to have, for some $1 \le i \le n$, $v_i = 0$ while $v_{-i} \neq 0$. These "degenerated indices" imply many disjoint cases, requiring to add the polynomials of type $H_{ijk}$. We thus suppose that $u$ admits a degenerated indice. There is then the following fact:

**Lemma 7.1.** There is a common $\delta = \pm$ such that for $1 \le i \le n$, $u_i = 0 \Leftrightarrow v_{\delta i} = 0$.
Proof. Take $i$ such that $u_i$ or $u_{-i}$ vanishes. Then $P_{ii}(u) = P_{ii}(v) = 0$ ensures that $v_i$ or $v_{-i}$ vanishes as well. Moreover, assume that we have $u_i \neq 0$ while $u_{-i} = 0$. Since $u$ belongs to a closed orbit, Lemma 4.3 claims that there exists some $1 \le t \le n$ such that $u_{-t} \neq 0$. Then $P_{it}(u) = P_{it}(v) \neq 0$ implies that $v_{\delta_{ii}} \neq 0$ and $v_{-\delta_{ii}} = 0$ for some $\delta_i = \pm$.

Take a second degenerated index $1 \le j \le n$ and let $\delta_j$ be the signus satisfying $u_j = 0 \Leftrightarrow v_{\delta_{jj}} = 0$. If $\delta_i \neq \delta_j$, we necessarily have $\begin{cases} P_{ij}(u) = 0 \\ P_{ij}(v) \neq 0 \end{cases}$ or $\begin{cases} P_{ij}(u) \neq 0 \\ P_{ij}(v) = 0 \end{cases}$. This is a contradiction since $P_{ij}(u) = P_{ij}(v)$. $\square$

The common sign $\delta$ provides the following lemmas:

**Lemma 7.2.** For any degenerated index $1 \le i \le n$, we have one of the rows of the following table:

<table><thead><tr><th>Vanishing coordinates</th><th>Defined quotient</th><th>$W_I^\delta = W_i^\delta \cap W_{-i}^\delta$</th><th>$W_I^{-\delta} = W_i^{-\delta} \cap W_{-i}^{-\delta}$</th></tr></thead><tbody><tr><td>$u_i$ and $v_{\delta_i}$</td><td>$c_{-i}^\delta$</td><td>$W_{-i}^\delta$</td><td>$\emptyset$</td></tr><tr><td>$u_{-i}$ and $v_{-\delta_i}$</td><td>$c_i^\delta$</td><td>$W_i^\delta$</td><td>$\emptyset$</td></tr></tbody></table>

Proof. Suppose that $\delta = +$ and $u_i = v_i = 0$. Then for any $z \in \mathbb{C}^*$ one has $z^{a_i}u_i = 0 = v_i$ while $z^{-a_i}u_i = 0 \neq v_{-i}$. This is $W_i = \mathbb{C}^*$ and $W_i^- = \emptyset$. We check that $W_I = \mathbb{C}^* \cap W_{-i} = W_{-i}$ while $W_I^- = \emptyset \cap W_{-i}^- = \emptyset$. Other cases of the table are obtained by similar considerations. $\square$

**Lemma 7.3.** Let $1 \le i < j \le n$ be integers. Then, $W_{IJ}^\delta \neq \emptyset$.

*Proof.* We first suppose that at $i$ or $j$ is degenerated. We then are in one of the rows of the following table:

<table><thead><tr><th>Vanishing coordinates</th><th>$P_{ij}(u)$</th></tr></thead><tbody><tr><td>$u_i, v_{\delta_i}$</td><td>$\neq 0$</td></tr><tr><td>$u_{-i}, v_{-\delta_i}$</td><td>$\neq 0$</td></tr><tr><td>$u_i, v_{\delta_i}, u_{-j}, v_{-\delta_j}$</td><td>$\neq 0$</td></tr><tr><td>$u_{-i}, v_{-\delta_i}, u_j, v_{\delta_j}$</td><td>$\neq 0$</td></tr><tr><td>$u_i, v_{\delta_i}, u_j, v_{\delta_j}$</td><td>0</td></tr><tr><td>$u_{-i}, v_{-\delta_i}, u_{-j}, v_{-\delta_j}$</td><td>0</td></tr></tbody></table>

We write for instance $\delta = +$, $u_i = v_i = 0$ but $u_j, u_{-j} \neq 0$. By Lemma 7.2, $W_I = W_{-i}$ while by Lemma 6.1, $W_J = W_j = W_{-j}$.

$$ \begin{align*} P_{ij}(v) = P_{ij}(u) &\Leftrightarrow p_{-ij}(u) = u_{-i}^{\varepsilon_{ij}} u_j^{\varepsilon_{ji}} = v_{-i}^{\varepsilon_{ij}} v_j^{\varepsilon_{ji}} = p_{-ij}(v) \\ &\Leftrightarrow c_{-i}^{\varepsilon_{ij}} = c_j^{\varepsilon_{ji}} \end{align*} $$

Then Lemma 3.3 claims that $W_{-ij} \neq 0$, and this gives $W_{IJ} \neq \emptyset$. The others cases with $P_{ij}(u) \neq 0$ are solved by similar considerations.

The cases with $P_{ij}(u) = 0$ need the polynomials of type $H_{ijk}$ to be solved. Assume for instance that $\delta = +$ and $u_i = u_j = v_i = v_j = 0$. Since $u$ belongs to a closed orbit, Lemma 4.3 ensures that there exists $1 \le k \le n$ with $u_k \neq 0$. The known cases of the table ensure that $W_{IK}$ and $W_{JK}$ are not empty. Then,

$$ \begin{align*} H_{ijk}(v) = H_{ijk}(u) &\Leftrightarrow u_{-i}^{q \varepsilon_{ijk}} u_{-j}^{r \varepsilon_{jik}} u_k^{(q' \varepsilon_{ijk} + r' \varepsilon_{jik})} = v_{-i}^{q \varepsilon_{ijk}} v_{-j}^{r \varepsilon_{jik}} v_k^{(q' \varepsilon_{ijk} + r' \varepsilon_{jik})} \\ &\Leftrightarrow (c_{-i}^q c_k^{q'})^{\varepsilon_{ijk}} = (c_{-j}^{-r} c_k^{-r'})^{\varepsilon_{jik}} \end{align*} $$

and Lemma 3.3 ensures that $W_{IJK} \neq \emptyset$. By inclusion $W_{IJ}$ is not empty. The other unknown rows of the table are solved by similar considerations.

We suppose now that $i$ and $j$ are not degenerated. By assumption, there is a third indice $k$ which is degenerated. We then have $u_k = v_{\delta_k} = 0$ or $u_{-k} = v_{-\delta_k} = 0$. We write for instance $\delta = +$ and $u_{-k} = v_{-k} =$
0. We already saw that in that case, $W_{IK}$ and $W_{JK}$ are not empty. Again the polynomials of type $H_{ijk}$ provide:

$$ H_{ijk}(v) = H_{ijk}(u) \iff \begin{cases} u_{-i}^{q} \varepsilon_{ijk} u_{-j}^{r} \varepsilon_{jik} u_{k}^{(q' \varepsilon_{ijk} + r' \varepsilon_{jik})} = v_{-i}^{q \varepsilon_{ijk}} v_{-j}^{r \varepsilon_{jik}} v_{k}^{(q' \varepsilon_{ijk} + r' \varepsilon_{jik})} \\ (c_{-i}^{q} c_{-j}^{q'})^{\varepsilon_{ijk}} = (c_{-i}^{-r} c_{-j}^{-r'})^{\varepsilon_{jik}} \end{cases} $$

and Lemma 3.3 ensures that $W_{IJK} \neq \emptyset$. By inclusion $W_{IJ}$ is not empty. The other cases are solved by similar considerations. □

We are now able to establish Theorem 7.4:

**Theorem 7.4.** The set of polynomials $\mathcal{F}$ separates closed orbits with respect to the action $O_2(\mathbb{C}) \curvearrowleft \hat{\mathcal{V}}$.

*Proof.* To prove it, we take $u$ and $v$ two vectors in closed orbits where polynomials of $\mathcal{F}$ agree and exhibit an element $g \in O_2(\mathbb{C})$ mapping $u$ to $v$. If none of the indices are degenerated, we refer to the proof of Theorem 6.5.

If there is some degenerated index, Lemma 7.3 claims that for all $1 \le i < j \le n$, $W_{IJ}^{\delta} \neq \emptyset$. Then, Corollary 3.5 completes in $\bigcap_{i=1}^{n} W_{I}^{\delta} \neq \emptyset$. Take now some $1 \le s \le q$, $i$ and $1 \le i \le n$ a degenerated index. Since

$u$ belongs to a closed orbit, Lemma 4.3 claims that there is some $1 \le j \le n$ such that $\begin{cases} u_i \neq 0 \Rightarrow u_{-j} \neq 0. \\ u_{-i} \neq 0 \Rightarrow u_{j} \neq 0. \end{cases}$

One has then $\begin{cases} p_{-ij}(u) \neq 0 \\ p_{-ji}(u) = 0 \end{cases}$ or $\begin{cases} p_{-ji}(u) \neq 0. \\ p_{-ij}(u) = 0. \end{cases}$ Anyway the polynomial $S_{ijs}$ provides

$$ S_{ijs}(u) = S_{ijs}(v) \iff \begin{cases} d_s(p_{-ij}(u) - p_{-ji}(u)) = d'_s(p_{-ij}(v) - p_{-ji}(v)) \\ d_s = \delta d'_s \end{cases} $$

This means that for any $z \in \bigcap_{i=1}^{n} W_{I}^{\delta}$, $\rho_{-1}(g_{z}^{\delta})(d_s) = d'_s$. Finally $\rho(g_{z}^{\delta})(u) = v$. □

## 7.2 Minimality

In general, the separating set of Theorem 7.4 is not minimal for the inclusion. We remove redundant polynomials to get a minimal separating set:

**Proposition 7.5.** In addition to those of Proposition 6.7, we make the following removals:

* For any $(i, j, k) \in \mathcal{T}(n)$ such that $a_k$ divides $a_i \lor a_j$, we remove $H_{ijk}$.
Then the separating set $\mathcal{F}$ is minimal for the inclusion.

*Proof.* We try to remove a polynomial and find two non separated closed orbits. For the polynomials of type $P, T, Q, D$ and $S$ we refer to section 6.

Assume that we remove $H_{ijk}$ for some $(i, j, k) \in \mathcal{T}(n)$. We take $u$ and $v$ satisfying one of the four unknown cases of Lemma 7.3 (otherwise, polynomials of type $P, T, Q, D$ and $S$ suffice to determine if $u$ and $v$ are in the same orbit). This is:

$$ \begin{cases} W_I^{\delta} \neq \emptyset \\ W_J^{\delta} \neq \emptyset \end{cases} \quad \text{and} \quad \begin{cases} W_I^{-\delta} = \emptyset \\ W_J^{-\delta} = \emptyset \end{cases} $$

If there exists another integer $l \neq k$ such that $P_{il}(u) \neq 0$, then by Lemma 7.3, $W_{IJ}^{\delta} \neq \emptyset$ and $u$ and $v$ are in the same orbit. Otherwise, by Lemma 4.3, $k$ is the only integer such that $P_{ik}$ and $P_{jk}$ do not vanish at $u$ and $v$. Then, by Lemma 7.3 we have $\begin{cases} W_{iK}^{\delta} \neq \emptyset \\ W_{jK}^{\delta} \neq \emptyset \end{cases}$. If $a_k$ divides $a_i \lor a_j$, Proposition 3.5 claims directly that $W_{IJK}^{\delta} \neq \emptyset$ and $u$ and $v$ lies in the same orbit.

On the opposite, if $a_k$ does not divide $a_i \lor a_j$, we choose $u$ and $v$ as in the following table:
<table><thead><tr><th></th><th>i</th><th>-i</th><th>j</th><th>-j</th><th>k</th><th>-k</th></tr></thead><tbody><tr><th>u</th><td>1</td><td>0</td><td>1</td><td>0</td><td>1</td><td>1</td></tr><tr><th>v</th><td>1</td><td>0</td><td>exp(<sup>2iπa<sub>j</sub></sup>&frasl;<sub>a<sub>k</sub></sub>)</td><td>0</td><td>1</td><td>1</td></tr></tbody></table>

These choices imply first that $W_I^- = W_J^- = \emptyset$. We next note $c_i = c_k = 1$ while $c_j = \exp(\frac{2i\pi a_j}{a_k})$. Then, $c_i^{\frac{a_i \vee a_j}{a_i \vee a_j}} c_j^{\frac{a_j \vee a_i}{a_j \vee a_j}} = \exp(\frac{2i\pi a_i}{a_k} \frac{a_i \vee a_j}{a_j}) = \exp(2i\pi \frac{a_i \vee a_j}{a_k}) \neq 1$. Lemma 3.3 provides by that way the intersections described in the right figure. We deduce that $P_{ik}$ and $P_{jk}$ agree at $u$ and $v$ while all the other polynomials in $\mathcal{F}$ vanish. Altough, $u$ and $v$ are not in the same orbit.

<figure><img src="image_5.png" /></figure>

**Remark 7.6.** The obtained minimal separating set thus has cardinality bounded by

$$n^2 \sim \underbrace{\frac{n(n+1)}{2}}_{P_{ij}} + \underbrace{\frac{m(m+1)}{2}}_{D_{st}} + \underbrace{m\binom{n}{2}}_{S_{ijs}} \le \#\mathcal{F} \le \underbrace{\frac{n(n+1)}{2}}_{P_{ij}} + \underbrace{3\binom{n}{3}}_{H_{ijk}} + \underbrace{3\binom{n}{3}}_{T_{ijk}} + \underbrace{3\binom{n}{4}}_{Q_{ijkl}} + \underbrace{\frac{m(m+1)}{2}}_{D_{st}} + \underbrace{m\binom{n}{2}}_{S_{ijs}} \sim n^4$$

The lower bound is reached when all the integers $a_i$ are equal to 1.

**Example 7.7.** Let us consider again the representation of $O_2(\mathbb{C})$ on $\hat{V} := \hat{V}_{-1}^3 \oplus \hat{V}_2 \oplus \hat{V}_3 \oplus \hat{V}_5 \oplus \hat{V}_7$. To the real separating set of cardinality 46 introduced in Example 6.11 are added the 12 polynomials of type H:

$$\left\{ \begin{array}{llllll} H_{2,3,1} & H_{1,2,3} & H_{1,3,2} & H_{2,4,1} & H_{1,2,4} & H_{1,4,2} \\ H_{3,4,1} & H_{1,3,4} & H_{1,4,3} & H_{3,4,2} & H_{2,3,4} & H_{2,4,3} \end{array} \right\}$$

Theorem 7.4 gives thus a separating set of cardinality 58. The difference of cardinality between real and complex separating set still exists but is less remarkable than for the representations of $SO_2(\mathbb{C})$ and $SO_2(\mathbb{R})$ (see Example 5.4).

# 8 Stratification of real representations by orbit type

Consider the representation of $O_2(\mathbb{R})$ on $\mathcal{V} := \bigoplus_{i=1}^m \mathcal{V}_{-1} \oplus \bigoplus_{i=1}^n \mathcal{V}_{a_i}$. [Azz+23] proved that since $O_2(\mathbb{R})$ is compact, each isotropy stratum is an algebraic subvariety $\overline{\Sigma_J} \subset \mathcal{V}$. In this section, we use the separating sets of Sections 5 and 6 to give algebraic equalities defining these strata.

**Definition 8.1.** Let $G$ be a compact group and $\rho$ a continuous representation on a vector space $\mathcal{V}$. Take $v$ a point in $\mathcal{V}$. The isotropy group of $v$ is defined as $\text{Stab}(v) = \{g \in G \mid \rho(g)(v) = v\}$. Two points are of same orbit type when their isotropy groups are conjugated. The isotropy stratum $\Sigma_J$ is the subset of points whose isotropy group is conjugated to $J < G$:

$$\Sigma_J = \{v \in \mathcal{V} \text{ such that } \text{Stab}(v) \in [J]\}$$

There is a partial order among isotropy classes. Namely, when $J_1, J_2$ are two isotropy groups, one notes

$$[J_1] < [J_2] \Leftrightarrow \exists K \in [J_1] \text{ such that } K < J_2$$

This induces a partial order among isotropy conjugacy classes, and one defines the isotropy stratum closure which corresponds to the set of points stabilized by at least a member of $[J]$:

$$\overline{\Sigma_J} = \bigcup_{[J]<[K]} \Sigma_K$$
The possible isotropy classes in SO₂(R) are [{I₂}], [SO₂(R)] itself and [Cₖ] the cyclic groups of order k ∈ N \ {0} (Figure 3). The possible isotropy classes in O₂(R) are [{I₂}], [SO₂(R)], [O₂(R)], [Cₖ] for k ∈ N \ {0} and [Dₖ] the dihedral groups of order 2k ∈ N* (Figure 4). The isotropy class [Cₖ] has a single representant: the subgroup of SO₂(R) generated by g_z⁺ where z is a kᵗʰ primitive root. However, the isotropy class [Dₖ] has an infinite number of conjugated representants. These are the subgroups of O₂(R) generated by Cₖ and a symmetry g_z⁻ ∈ O₂(R).

<figure><img src="image_6.png" /><figcaption>Figure 3</figcaption></figure>

<caption>Figure 4</caption>

The decomposition into irreducible components helps to compute the stabilizer of a point. The isotropy group of $v \in V$ is the intersection of the isotropy groups of its projections onto each representations in the irreducible decomposition [Oli14]. This leads to the following Theorem:

**Theorem 8.2.** Consider the action of SO₂(R) on V. Let v be a point in V.

* For $k \in N \setminus \{0\}$, $v \in \overline{\Sigma_{C_k}}$ iff for all $i$ such that $k$ does not divide $a_i$, $$ \Re(p_{-ii})(v) = 0 $$ $$ \Im(p_{-ii})(v) = 0 $$

* $v \in \overline{\Sigma_{SO_2(\mathbb{R})}}$ iff for all $i \le n$, $$ \Re(p_{-ii})(v) = 0 $$ $$ \Im(p_{-ii})(v) = 0 $$

*Proof.* For any $k \in N^*$, the possible isotropy classes in $V_k$ are SO₂(R) and Cₖ, and the only point of isotropy class SO₂(R) is (0,0). By intersection then, the isotropy group Stab(v) = C<sub>ϕ(v)</sub>, where ϕ(v) is the gcd of all $a_i$ such that the projection of v onto $V_{a_i}$ is not zero, or equivalently $p_{-ii}(v) \neq 0$. If there is no such $a_i$, then Stab(v) = SO₂(R). □

**Theorem 8.3.** Consider the action of O₂(R) on V. Let v be a point in V.

* For any $k \in N \setminus \{0\}$, $v \in \overline{\Sigma_{C_k}}$ iff for all $1 \le i \le n$ such that $a_i$ does not divide k, $P_{ii}(v) = 0$.

* For any $k \in N \setminus \{0\}$, $v \in \overline{\Sigma_{D_k}}$ iff $$ \begin{cases} \forall 1 \le i \le n \mid a_i \text{ does not divide } k, P_{ii}(v) = 0. \\ \forall 1 \le i < j \le n, (P_{ij}^2 - 4P_{ii}^{\epsilon_{ij}} P_{jj}^{\epsilon_{ji}})(v) = 0. \\ \forall 1 \le s \le q, D_{ss}(v) = 0. \end{cases} $$

* $v \in \overline{\Sigma_{SO_2(\mathbb{R})}}$ iff for all $i \le n$, $P_{ii}(v) = 0$.

* $v \in \overline{\Sigma_{O_2(\mathbb{R})}}$ iff for all $i \le n$, $P_{ii}(v) = 0$ and for all $1 \le s \le q$, $D_{ss}(v) = 0$.

*Proof.* We look at the possible isotropy classes of the projection on each irreducible representation:

* For any $k \ge 2$, the possible isotropy classes in $V_k$ are O₂(R) and Dₖ, and the only point of isotropy class O₂(R) is (0,0).

* The possible isotropy classes in $V_{-1}$ are SO₂(R) and O₂(R), and the only point of stabilizer O₂(R) is 0.

We first identify the subset $\text{Stab}(v)^+ = \text{Stab}(v) \cap \text{SO}_2(\mathbb{R})$. As for Theorem 8.3, $\text{Stab}(v)^+ = C_{\varphi(v)}$, where $\varphi(v)$ is the gcd of all $a_i$ such that the projection of v onto $V_{a_i}$ is not zero, equivalently $P_{ii}(v) \neq 0$. If there is no such $a_i$, then $\text{Stab}(v)^+ = \text{SO}_2(\mathbb{R})$.

We next determine if there exists a symmetry $g_z^- \in \text{Stab}(v)$. If not, $\text{Stab}(v) = \text{Stab}(v)^+$. If yes, $\text{Stab}(v) = \text{Stab}(v)^+ \times C_2$. There exists $z \in \mathbb{C}^*$ such that $\rho(g_z^-)(v) = v$ iff $\bigcap_{i=1}^n W_I^- \neq \emptyset$ and $\forall 1 \le s \le m$, $d_s = 0$. We
then check that for all $1 \le i < j \le n$,

$$ (P_{ij}^2 - 4P_{ii}^{\epsilon_{ij}} P_{jj}^{\epsilon_{ji}}) (v) = 0 \quad \Leftrightarrow \quad \begin{aligned} & v_{-i}^{2\epsilon_{ij}} v_j^{2\epsilon_{ji}} + 2(v_{-i}v_i)^{\epsilon_{ij}} (v_{-j}v_j)^{\epsilon_{ji}} + v_i^{2\epsilon_{ij}} v_{-j}^{2\epsilon_{ji}} - 4(v_{-i}v_i)^{\epsilon_{ij}} (v_{-j}v_j)^{\epsilon_{ji}} = 0 \\ & v_{-i}^{2\epsilon_{ij}} v_j^{2\epsilon_{ji}} - 2(v_{-i}v_i)^{\epsilon_{ij}} (v_{-j}v_j)^{\epsilon_{ji}} + v_i^{2\epsilon_{ij}} v_{-j}^{2\epsilon_{ji}} = 0 \\ & (p_{-ij} - p_{-ji}) (v)^2 = 0 \\ & W_{IJ}^{\infty} \neq \emptyset \end{aligned} $$

Furthermore we have $\forall 1 \le s \le m, d_s = 0 \Leftrightarrow \forall 1 \le s \le m, D_{ss}(v) = 0$. □

**Remark 8.4.** Altough the groups $C_2$ and $D_1$ are isomorphic, they are not conjugated in $SO_2(\mathbb{R})$. They thus represent two distinct isotropy classes, as enumerated in Theorem 8.3. The corresponding strata form two distinct irreducible varieties in the orbit space.

**Example 8.5.** Consider the representation of $SO_3(\mathbb{R})$ on the space of piezoelectricity tensors Piez. [HJ24] identifies a Seshadri slice which reduces to the action of $O_2(\mathbb{R}) < SO_3(\mathbb{R})$ on a subvariety $S \subset$ Piez. This representation of $O_2(\mathbb{R})$ decomposes in

$$ S = V_{-1}^3 \oplus V_0 \oplus V_1^3 \oplus V_2^2 \oplus V_3. $$

The stratification of the orbit space $S/O_2(\mathbb{R})$ allows to deduce the isotropy group of generic tensors in Piez. This stratification is given by:

* $v \in \overline{\Sigma_{C_2}} \Leftrightarrow P_{11}(v) = P_{22}(v) = P_{33}(v) = P_{66}(v) = 0$.

* $v \in \overline{\Sigma_{C_3}} \Leftrightarrow P_{11}(v) = P_{22}(v) = P_{33}(v) = P_{44} = P_{55}(v) = 0$.

* $v \in \overline{\Sigma_{D_1}} \Leftrightarrow \begin{cases} \forall 1 \le i < j \le 6, (P_{ii}^{\epsilon_{ij}} P_{jj}^{\epsilon_{ji}} - P_{ij}^2) (v) = 0. \\ D_{11}(v) = D_{22}(v) = D_{33}(v) = 0 \end{cases}$

* $v \in \overline{\Sigma_{D_2}} \Leftrightarrow \begin{cases} P_{11}(v) = P_{22}(v) = P_{33}(v) = P_{66}(v) = 0 \\ (P_{44}P_{55} - P_{45}^2) (v) = 0 \\ D_{11}(v) = D_{22}(v) = D_{33}(v) = 0 \end{cases}$

* $v \in \overline{\Sigma_{D_3}} \Leftrightarrow \begin{cases} P_{11}(v) = P_{22}(v) = P_{33}(v) = P_{44}P_{55}(v) = 0 \\ D_{11}(v) = D_{22}(v) = D_{33}(v) = 0 \end{cases}$

* $v \in \overline{\Sigma_{SO_2(\mathbb{R})}} \Leftrightarrow P_{11}(v) = P_{22}(v) = P_{33}(v) = P_{44}(v) = P_{55}(v) = P_{66}(v) = 0$.

* $v \in \overline{\Sigma_{O_2(\mathbb{R})}} \Leftrightarrow \begin{cases} P_{11}(v) = P_{22}(v) = P_{33}(v) = P_{44} = P_{55}(v) = P_{66}(v) = 0 \\ D_{11}(v) = D_{22}(v) = D_{33}(v) = 0 \end{cases}$

## References

[Azz+23] Perla Azzi, Rodrigue Desmorat, Julien Grivaux, and Boris Kolev. “Rationality of normal forms of isotropy strata of a representation of a compact Lie group”. In: arXiv preprint arXiv:2301.08599 (2023).

[BV23] B Blum-Smith and S. Villar. “Machine learning and invariant theory”. In: Notices of the American Mathematical Society (2023).

[Bür+21] P. Bürgisser, M. Doğan, V. Makam, M. Walter, and A. Wigderson. “Polynomial time algorithms in invariant theory for torus actions”. In: arXiv preprint arXiv:2102.07727 (2021).

[DK15] H. Derksen and G. Kemper. *Computational invariant theory*. 2nd. Springer-Verlag, 2015.

[Des+23] Boris Desmorat, Marc Olive, Nicolas Auffray, Rodrigue Desmorat, and Boris Kolev. “Computation of minimal covariants bases for 2D coupled constitutive laws”. In: International Journal of Engineering Science 191 (2023), p. 103880.

[Duf09] Emilie Dufresne. “Separating invariants and finite reflection groups”. In: Advances in Mathematics 221.6 (2009), pp. 1979–1989.
[DG24] Nadav Dym and Steven J Gortler. "Low-Dimensional Invariant Embeddings for Universal Geometric Learning". In: *Foundations of Computational Mathematics* (2024), pp. 1-41.

[GSS88] M. Golubitsky, I. Stewart, and D. Schaeffer. *Singularities and groups in bifurcation theory. Vol. II*. Vol. 69. Applied Mathematical Sciences. New York: Springer-Verlag, 1988.

[HJ24] Evelyne Hubert and Martin Jalard. "Algebraically independent generators for the invariant field of $SO(3)$ (R) and $O(3)$ (R) representations $R \oplus H$". working paper or preprint. June 2024. URL: https://inria.hal.science/hal-04604969.

[Kem24] Gregor Kemper. "Invariant Theory: a Third Lease of Life". In: *arXiv preprint arXiv:2403.12709* (2024).

[MFK94] David Mumford, John Fogarty, and Frances Kirwan. *Geometric invariant theory*. Vol. 34. Springer Science & Business Media, 1994.

[Oli14] M. Olive. "Géométrie des espaces de tenseurs Une approche effective appliquée à la mécanique des milieux continus". PhD thesis. AMU, 2014.

[Ser66] Jean Pierre Serre. *Algebres de Lie semi-simples complexes*. New York : Amsterdam : W. A. Benjamin, 1966.

[Stu08] Bernd Sturmfels. *Algorithms in invariant theory*. Springer Science & Business Media, 2008.

[Wey46] H Weyl. *The classical groups: their invariants and representations*. Vol. 45. Princeton university press, 1946.