Riemannian Manifold Learning for Stackelberg Games with Neural Flow Representations

Authors: Larkin Liu, Yutong Chao, Jalal Etesami, Kashif Rasul
Primary area: learning on graphs and other geometries & topologies
Keywords: Neural Normalizing Flows, Stackelberg Games, Riemannian Manifolds

Abstract:
We present a novel framework for online learning in Stackelberg general-sum games, where two agents, the leader and follower, engage in sequential turn-based interactions. At the core of this approach is a learned diffeomorphism that maps the joint action space to a smooth Riemannian manifold, referred to as the $\textit{Stackelberg manifold}$. This mapping, facilitated by neural normalizing flows, ensures the formation of tractable isoplanar subspaces, enabling efficient techniques for online learning. By assuming linearity between the agents' reward functions on the $\textit{Stackelberg manifold}$, our construct allows the application of standard bandit algorithms. We then provide a rigorous theoretical basis for regret minimization on convex manifolds and establish finite-time bounds on simple regret for learning Stackelberg equilibria. This integration of manifold learning into game theory uncovers a previously unrecognized potential for neural normalizing flows as an effective tool for multi-agent learning. We present empirical results demonstrating the effectiveness of our approach compared to standard baselines, with applications spanning domains such as cybersecurity and economic supply chain optimization.

Decision: reject
Reviewer scores: 6, 5, 6, 6

Reviews and discussion:
<thread>
creation date:
1729444312402

thread_type:
official_review

<thread_object>
creator: reviewer_ngcu

<content>
summary:
1. **Summary:** This paper presents a novel framework for online learning in Stackelberg general-sum games, which involves two agents: a leader and a follower. The key innovation is the use of a learned diffeomorphism that maps the joint action space to a smooth Riemannian manifold, termed the Stackelberg manifold. This approach, facilitated by neural normalizing flows, creates isoplanar subspaces that simplify the application of standard bandit algorithms for online learning. The authors assume linearity of the agents' reward functions on the Stackelberg manifold, providing a theoretical foundation for regret minimization on convex manifolds and establishing finite-time bounds on simple regret for learning Stackelberg equilibria. The paper demonstrates the effectiveness of this manifold learning integration into game theory, showing its potential for neural normalizing flows as a tool for multi-agent learning. Empirical results are presented, comparing the approach to standard baselines and showing improvements in computational efficiency and regret minimization across applications in cybersecurity and economic supply chain optimization.

2. **Contribution:** The paper's contributions are significant in advancing the understanding of Stackelberg learning under imperfect information and providing a rigorous framework for optimizing Stackelberg games on spherical manifolds.

strengths:
The primary strengths of this paper can be summarized in the following two aspects:

1. The paper presents a novel approach of mapping actions to a Riemannian manifold to solve the Stackelberg equilibrium.
2. The paper provides both rigorous theoretical analysis and comprehensive empirical validation.

weaknesses:
Here are the major weaknesses from my perspective:

1. The authors have not compared their method to many current state-of-the-art approaches for solving Stackelberg equilibrium. The only comparison provided is in the experimental section, where they benchmark their algorithm against the dual-UCB method. The lack of broader comparisons with other recent algorithms, both from an empirical and theoretical standpoint, limits the contribution.

2. As a minor suggestion, the author should provide additional background information on Riemannian manifolds, as the concept may not be familiar to the broader deep learning community.


Additionally, I have a minor suggestion: I recommend that the authors switch to the official template for the conference to ensure a more formal presentation.

questions:
Based on the discussion of the paper's strengths and weaknesses, I have the following questions for the authors:

1. What is the intuition behind mapping the action space to a Riemannian manifold? Does this approach stem from leveraging methods such as Riemannian gradient descent?
2. Is it necessary to consider a general Riemannian manifold, or is it sufficient to focus only on the unit sphere?
3. Can the authors provide a theoretical explanation of why their algorithm has advantages over previously proposed methods?
</content>
</thread_object>

</thread>
<thread>
creation date:
1729618770887

thread_type:
official_review

<thread_object>
creator: reviewer_d4pn

<content>
summary:
This work proposes an approach to no-regret learning in Stackelberg games that first embeds the original joint actions in a Riemmanian manifold, in particular the sphere. The authors deploy techniques from normalizing flows to learn this invertible mapping using a loss function they designed to approximately satisfy several desiderata (D1-D6) that they argue a *Stackelberg Embedding* should satisfy. They then propose an algorithm that takes advantage of this embedding to estimate optimal *embedding* actions that they then map back to the original action space. Experiments show the regret of their algorithm (GISA) grows more slowly than the baseline UCB algorithm.

strengths:
The idea of learning an embedding of a Stackelberg game such that the game can be reasoned about and solved in that space more easily is novel and interesting. To accomplish this, the authors leverage normalizing flows; in general, this work requires aggregating ideas and techniques from several typically disparate research areas which makes it original. While the paper focuses on the spherical manifold, the proposed loss function in equation 2.14 provides a way of learning the embedding, not just for spherical manifolds.

weaknesses:
As the paper is currently presented, there was little discussion or motivation early on for why a spherical embedding (or any embedding) would be helpful for constructing an improved no-regret algorithm. The learned mapping is invertible so there's no dimensionality reduction, which is a typical motivation in other work making similar assumptions. Section 2.3 is introduced eventually, but it would seem the assumption of reward functions that are linear in the embedding space is critical for this approach to be strong. For example, in the linear MDP literature, this reward assumption is stated early on as part of the problem definition [1].

It's unclear how the precise experimental settings illustrate the importance of the embeddings. Do the reward functions in this game satisfy the linearity assumption in the chosen embedding space?

[1] Jin, Chi, et al. "Provably efficient reinforcement learning with linear function approximation." Conference on learning theory. PMLR, 2020.

questions:
In general, I like the idea of the paper but I feel the writing needs to be substantially improved (e.g., motivation and layout of paper mentioned in weaknesses above). I also feel the experimental domains could be supplemented with a few toy domains that are tailored to the specific reward assumption for a given embedding. Including additional manifolds beyond the sphere manifold would strengthen the paper as well if you can construct meaningful motivations for them.

- Figure 1: I originally found Figure 1 to be saying something contradictory to Lemmas 4.3 and 4.4. The red arrow corresponding to $\\Phi_a$ appeared to be suggesting that $\\Phi_a$ runs horizontally. Similarly, $\\Phi_b$ points to a horizontal surface. Can you change $\\Phi_a$ to simply point to a single longitudinal line? Also, maybe make $\\Phi_b$'s arrow blue to be consistent?
- Algorithm 1: line 12 shows the algorithm returning an action before the for-loop completes. Is there an indent typo here? Do you mean *yield* rather than *return*? Also, line 12 states "return $a^*$" whereas I assume you mean $\\hat{a}^*$.
- Figure 2 / Algorithm 1: Presumably, these two can help the reader better understand the algorithm. Figure 2 uses $\\theta'\_B$ while Algorithm 1 uses $\\hat{\\theta}\_B$. Can these be made consistent? Also, if the green circle represents $\\mathcal{B}(\\theta, \\hat{\\theta}\_B)$, can you point out $\\tilde{\\theta}\_A$ specifically in Figure 2?
- Theorem 1: Is there a typo? "and Eq. (2.15) and Eq. (2.16)" is repeated. I'm not sure the sentence currently makes sense either. Do you mean to say "and the linear relationship to the reward function.... holds"?
- Section 4.1, near end of first paragraph: "In the Stackelberg game, since the follower moves first...". You mean the leader moves first.
- Paragraph "Mapping to a Spherical Manifold": The sentence "The transformation from spherical coordinates to Cartesian coordinates is used to map input features onto a $D$-dimensional spherical manifold" seems to read backwards. It should be "transformation from Cartesian to spherical..." or the "inverse of the transformation from...".
- How might you handle Stackelberg games with discrete action spaces?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730731896342

thread_type:
official_review

<thread_object>
creator: reviewer_r8sr

<content>
summary:
The paper studies Stackelberg games where a leader and follower take actions sequentially. The framework presented in the paper maps the joint actions pace to a smooth Riemannian manifold for efficient methods for online learning. It assumes the agents' rewards function are linear on the Stackelberg manifold and then uses bandit algorithms.

strengths:
The paper tries to provide a framework to map the Stackelberg games to a bandit problem by using neural normalizing flows and by assuming the reward function is linear on the mapped manifold, allowing the use of the bandit results to study the game. The paper could provide regret analysis and empirical results. 

However, the reviewer is very confused by the motivation of the approach. More details of weakness is shown below.

weaknesses:
The paper is not very well structured and not very motivated. The contributions are not clear. The problem is statement is not clearly stated. It seems like the paper considers a 2-player Stackelberg game and considers bounding a form of regret as a measure of performance. However, Section 2.2 describes a procedure to obtain an embedding on the D-sphere where the reward functions are linear as in (2.16). First of all, it is unclear why people should do this embedding (motivation is not clear). Secondly, It is not clear how such an embedding is possible. The loss in 2.14 does not reflect that. 

There are several lemmas and theoretical results in the paper that do not seem to hold as currently stated and/or the provided proof doesn’t make sense. Examples include:

1) Definition 2: it seems like such a manifold does not exist unless it is a singleton, because any two distinct points as a subset is certainly not geodetically convex.
2) Lemma 4.1 where the statement does not make sense for any non-compact domain.
3) Lemma 4.3: the justification of this lemma does not make sense other than restating the Poincare-Hopf Theorem.

In the simulations, the rewards models seem to be linear in \theta parameters automatically, then it is not clear what is the purpose of Section 2.2. regarding the embedding.

questions:
see comments under the weakness section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730790270615

thread_type:
official_review

<thread_object>
creator: reviewer_cjwa

<content>
summary:
This paper studies the Stackelberg general-sum games.  They propose a learned diffeomorphism that maps the joint action space to the so-called Stackelber Manifold using neural normalizing flows. This way, they convert the equilibrium learning problem into an online learning problem on manifolds. Applying multi-armed bandit methods, this proposed framework is shown to achieve efficient computation of the equilibrium. Simulation results are provided to validate the proposed methods.

strengths:
The proposed framework is highly novel to me. It effectively bridges equilibrium learning, manifold learning, and online learning paradigms by establishing theoretical and practical connections. The application of normalizing neural flows shows particular promise for converting equilibrium learning problems into online manifold learning, and optimization problems, opening valuable new research directions.

weaknesses:
The paper would benefit from a clearer organizational structure, particularly in Section 4. The progression of ideas through Lemmas 4.1-4.4 needs stronger motivation and explicit statements of their individual contributions to the overall narrative. Additionally, the technical foundation could be strengthened by providing clearer connections between consecutive lemmas and their role in building the theoretical framework and by including fundamental definitions from Riemannian geometry, especially the concept of geodesics, which is central to the framework. In summary, I hope the authors make fewer assumptions about the reader's background knowledge in differential geometry.

questions:
- Could the authors discuss more on the roles of Lemmas 4.1-4.4?
- Could the authors explain what structures of Riemannian manifolds are essential in the proposed framework? Can this framework be applied to space with fewer structures, say a smooth differential manifold?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732021654933

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response (cont.)

comment:
### **General Response 4:** Significance of the Riemannian and Spherical Manifold  

**Riemannian vs. Smooth Manifold:**  The selection of a Riemannian manifold (RM) for our study offers several key advantages. Notably, the RM enforces a consistent distance metric on any smooth manifold, allowing for a rigorous definition of geodesic distances. Operators such as the gradient are also well-defined on the topology. While this formalism adds security, Riemannian gradient descent [7] could be applied when less convenient manifold geometries (e.g., non-spherical) are imposed.  

If we were to work with smooth manifolds alone, this would not pose severe problems for Lemmas 4.1–4.4 and Theorem 1 when dealing with a D-sphere. However, challenges could arise when extending the theory to non-spherical manifolds (e.g., hyperbolic spheres, ellipsoids). For such extensions, it is crucial to maintain a Euclidean metric for geodesic distances. Nevertheless, in this paper, we focus on establishing guarantees for the D-dimensional spherical manifold. The use of RM tooling ensures robustness against potential issues in future extensions of our framework.  

**Non-Spherical Manifolds:** Some of the basic theory in our work could, in principle, extend to other types of manifolds, such as hyperbolic spheres or ellipsoids. However, this is not the primary focus of our paper. The following discussion regarding applications to non-spherical manifolds serves to enrich the framework and encourage future exploration. Specifically, Lemma 4.2 will hold for any convex manifold. Lemmas 4.3–4.4 will hold due to the Poincaré-Hopf theorem and manifold compactness.  

However, Lemma 4.1 and Theorem 1 may require adjustments for non-spherical manifolds. For Lemma 4.1, we must ensure the geodesic distance between any two projected points on the manifold is proportional to the divergence angle formed by the objective vectors $\theta_A'$ and $\theta_B'$. If this is not strictly the case, Lemma 4.1 should be modified to include an alternative geodesic distance measure or orientation.  

Theorem 1 is designed for the spherical manifold, as it is convenient to draw azimuthal lines tangent to the confidence projected onto $\mathbf{\Phi}$ as convex submanifolds (illustrated in Fig. 2). While a similar argument could be made for non-spherical manifolds, it would not follow directly from Theorem 1 and would require adjustments to how confidence regions are projected and subspaces are formed with respect to these regions. Nevertheless, the key contribution of our work is demonstrating how a complex game-theoretic equilibrium computation problem can be transformed into a simpler topological geometry problem, with the D-sphere serving as the pioneering example.

---
[1] Amani, Sanae, Mahnoosh Alizadeh, and Christos Thrampoulidis (2019). “Linear stochastic bandits under safety constraints”. In: Advances in Neural Information Processing Systems 32.

[2] Moradipari, Ahmadreza et al. (2022). “Feature and parameter selection in stochastic linear bandits”. In: Inter- national Conference on Machine Learning. PMLR, pp. 15927–15958.

[3] Zanette, Andrea et al. (2021). “Design of experiments for stochastic contextual linear bandits”. In: Advances in Neural Information Processing Systems 34, pp. 22720–22731.

[4] Brehmer, Johann and Kyle Cranmer (2020). “Flows for simultaneous manifold learning and density estimation”. In: Advances in neural information processing systems 33, pp. 442–453.

[5] Durkan, Conor et al. (2020). nflows: normalizing flows in PyTorch. Version v0.14. DOI: 10.5281/zenodo. 4296287. URL: https://doi.org/10.5281/zenodo.4296287.

[6] Dinh, Laurent, Jascha Sohl-Dickstein, and Samy Bengio (2016). “Density estimation using real nvp”. In: arXiv preprint arXiv:1605.08803.

[7] Bonnabel, Silvere. "Stochastic gradient descent on Riemannian manifolds." IEEE Transactions on Automatic Control 58.9 (2013): 2217-2229.

[8] Liu, Larkin, and Yuming Rong. "No-Regret Learning for Stackelberg Equilibrium Computation in Newsvendor Pricing Games." arXiv preprint arXiv:2404.00203 (2024).

[9] Roughgarden, Tim. "Algorithmic game theory." Communications of the ACM 53.7 (2010): 78-86.

[10] Lattimore, Tor, and Csaba Szepesvári. Bandit algorithms - Chapter 19. Cambridge University Press, 2020.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732021947782

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response (cont.)

comment:
### **General Response 2:** Intuition & Justification of Lemmas 4.1–4.3  

**Intuition of Lemma 4.1:**  The divergence angle between any two objective vectors can be made proportional to the geodesic distance between the extension of the objective vector onto $\mathbf{\Phi}$. The use case here is quantifying the best response (BR) under a constrained space. The smaller the divergence angle between $\theta_A$ and $\theta_B$, the more preferable it is for Agent B (assuming $\theta_A$ is fixed). Given a constrained space $\overleftrightarrow{\mathbf{\Phi}}_{\mathbf{a}}$, Agent A (the leader) can anticipate what a rational follower would do by examining the geometry of the embedding space and the uncertainty surrounding $\theta_B$. Both agents will try to position the joint action on the manifold closest to their respective objective vectors $\theta_A$ and $\theta_B$. This new method for computing the BR under uncertainty allows us to greatly simplify the BR computation, which is crucial for calculating Stackelberg equilibria.  

**Intuition of Lemma 4.2:** We argue that on any strictly convex manifold, there could exist only one unique reward-maximizing solution for each objective vector and convex subspace. This simplifies the search for the equilibrium location, as there is a unique point. This is termed a pure strategy in the existing game theory literature [8], as opposed to a mixed strategy (which arises in situations where multiple equilibria could exist).  

**Intuition of Lemma 4.3:** The purpose of Lemma 4.3 is to highlight that, given the bipartite map in Definition 2.1, subspaces are guaranteed to intersect on the manifold. This is easy to visualize on a spherical manifold in the 2-sphere setting (e.g., longitudinal and latitudinal lines) but becomes challenging to intuit in higher dimensions. We rigorously argue that, just as in the 2-sphere case, the same principle holds in a D-sphere setting.  

### **General Response 3:** Advantages of Online Learning on a Manifold  

Provided the bipartite spherical map in Definition 2.1, when the agents map their joint actions to $\mathbf{\Phi}$ (or the D-sphere surface), the leader induces subspaces that simplify the follower’s best response region, both under perfect information and quantifiable uncertainty. This can be exploited to derive the high-probability worst-case outcome at each turn, based on the uncertainty of the parameter estimates for $\hat{\theta}_A$ and $\hat{\theta}_B$. This allows us to provide theoretical guarantees on the simple regret of the algorithm, which follows from Theorem 1.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732022185998

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response

comment:
### **General Response 1:** Linear Relationship of the Embedding Space $\mathbf{\Phi}$ to the Reward Space:

We affirm that there exists a linear relationship between the embedding space and the reward space. The linear bandit assumption—that the expected reward is the inner product between an embedding space and some linear parameters—is commonly accepted in the online learning literature [1, 2, 3, 10], as cited in our draft Section 2.2.2.  To demonstrate rigorously, let $Y = \mathbf{E}[\mu]$. Given a linear relation in the action space $X$, the reward can be expressed as:  
$$
Y = \langle \tilde{\theta}, \tilde{\phi}(X) \rangle.
$$
This tool, $\tilde{\phi}(\cdot)$, is known as a feature map [3]. From one perspective, we can view the linear feature map as the final layer in a neural network output. Instead of learning the action-to-reward relation directly via a neural network, we learn the relation of the ambient space (or the joint action space) to the embedding space.

**Theoretical Argument:** Given $Y = \mathbf{E}[\mu]$ and a linear relation in the action space $X$, under some mild assumptions, we want to show that there always exist some alternative parameters $\theta$ and $\phi$ such that:  
$$
\langle \tilde{\theta}, \tilde{\phi}(X) \rangle = \langle \theta, \phi(X) \rangle.
$$  

Suppose we assume the existence of a bijective transformation $T: \tilde{\phi}(X) \mapsto \phi(X)$, and its inverse $T^{-1}: \phi(X) \mapsto \tilde{\phi}(X)$, which is smooth and differentiable with respect to $X$. We can then express:  
$$
Y = \langle \tilde{\theta}, T^{-1}(\phi(X)) \rangle.
$$  
To swap from $\tilde{\theta}$ to $\theta$, we select $\theta$ as:  
$$
\theta = (J_T^\top)^{-1} \tilde{\theta},
$$  
where $J_T$ is the Jacobian of $T(\cdot)$ with respect to $\tilde{Y}$, where $\tilde{Y} \in \text{Im}(\tilde{\phi})$. Therefore, we show the existence of $\theta$ such that:  
$$
Y = \langle \theta, \phi(X) \rangle,
$$  
where $\phi(X) \equiv T \circ \tilde{\phi}(X)$. This argument relies on the bijectiveness and smoothness of $T(\cdot)$, consequently being a diffeomorphism, a fair assumption as long as we consider $\tilde{\phi}(X)$ to be smooth and well-behaved. For non-smooth $\tilde{\phi}(X)$, we could settle for a continuous-to-discrete approximation, particularly useful for large discrete action spaces.  

Since there are no limitations to what the learned feature map $\hat{\phi}$ can be, it offers great flexibility in constructing $\hat{\phi}$ to approximate $T \circ \tilde{\phi}(\cdot)$. For this purpose, we apply normalizing flows [4, 5, 6], a recent technology developed specifically for mapping ambient data to a desired manifold, $\mathbf{\Phi}$, of which multiple sufficient approximations for $\hat{\phi}$ could potentially exist. This enables the construction of $\mathbf{\Phi}$ such that the linear relation between the embedding space and reward space holds.

###  **General Response 1A:** Simulation Environments & Linear Structure of Reward Function  
**Simulation Environments:** We present three games (R1, NPG, and SSG) where simulations were run to produce empirical results. The R1 and SSG games are both linear by design, as per the structure of Eq. G.1, G.2, and Eq. G.12 in Appendix G. This directly aligns with our reward function structure from (Eq. 2.16).  

The NPG game is taken from the work of [8], representing a realistic supply chain application. Here, the reward function is multiplicative and additive, operating in 1D and 2D continuous space. In theory, any feature map can extend additional dimensions to the multiplicative factors to capture the multiplicative causal variables. This can be implicitly done via normalizing flows.  

Nevertheless, knowing that there is an underlying non-linear to linear structure to the reward function defeats, at least in part, the motivation for our work. The key idea behind $\mathbf{\Phi}$ is to abstract away the need to know the exact reward structure, as there could be many embeddings in existence that could work under our proposed framework. We are able to select freely the architecture of the normalizing flow neural network, as well as increase the dimensionality of the spherical manifold (or the output dimension), to learn a sufficient embedding.  

Given this freedom of architectural choice, we aim to learn an embedding that is well-behaved following the desiderata from D1–D6 in Table 1. The problem solver may not necessarily know the structure of the problem beforehand, and perhaps there exists only a set of offline data to begin with, where they will construct the best approximate model for the ideal Stackelberg embedding. As online data arrives, the model could, in principle, be retrained to produce a new feature map. This does not affect the procedure of Algorithm 1 (GISA).
</content>
</thread_object>

</thread>
<thread>
creation date:
1732022431910

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
We thank all of the reviewers deeply for their thoughtful and in-depth reviews of our draft. To efficiently address all concerns, we provide a General Response section, which serves as an additional discussion and clarification of points raised by multiple reviewers that overlap in terms of topic. We hope that the additional discussion will further improve the understanding of our work and resolve outstanding questions.

As suggested by some of the reviewers, we have also revised the manuscript with relatively minor updates addressing typos, minor adjustments to diagrams, adjustments to Definitions & Lemmas, and an update to the ICLR 2025 template.

---
</content>
</thread_object>

</thread>
<thread>
creation date:
1733242978089

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Authors' Response to Reviewer Feedback: Revisions and Key Contributions

comment:
Once again, we deeply extend our gratitude to all the reviewers for their time and meaningful suggestions, many of which have been incorporated into this revision.

Our work addresses an inherently challenging problem, bridging multi-agent learning and manifold geometry. By introducing a novel connection between these domains using normalizing flows, we enable more efficient game-theoretic online learning in Stackelberg games compared to conventional solutions such as bi-level optimization.

In response to the feedback received, we have improved the clarity of our exposition to make the text accessible to a broader audience. We are especially grateful for the detailed feedback from positive reviewers, which has significantly enhanced the quality of our paper. Unfortunately, reviewer R8Sr, who raised strong initial concerns regarding our work, was unable to participate further during the discussion period, but we appreciate their useful input nonetheless. We believe our rebuttal and revisions have largely addressed those concerns, particularly those related to theoretical issues surrounding the manifold geometry.

We are pleased that the rebuttal and discussion have been very constructive, resulting in a strengthened assessment and an overall increase in the reviewers’ scores for our paper.

To highlight our core contributions again, we primarily focus on:

1. Providing a technique that enables the learning of a well-behaved manifold, $\mathbf{\Phi}$, via normalizing flows, as detailed in Section 2.2.2.

2. Establishing a rigorous theoretical foundation for our manifold learning technique. This learned diffeomorphism  reveals the linear relationship between the embedding space and the joint reward function, enabling straightforward Stackelberg regret guarantees.

Complementing our theory, we provide empirical results from realistic game-theoretic simulations, avoiding reliance on toy examples. Our extensible framework lays a foundation that can be adapted to future applications across diverse learning environments and alternative Riemannian manifolds. We consider our normalizing flow technique and theoretical guarantees in the context of multi-agent learning to be key contributions to the machine learning community.

We hope this summary effectively highlights the contributions and significance of our work. As always, we are grateful for the reviewers’ thoughtful ideas and suggestions, and we remain open to future refinements to our paper.
</content>
</thread_object>

</thread>


Metareview:
metareview:
Summary:
This paper investigates learning Stackelberg games with large, potentially infinite, action spaces. To address the challenge of large action spaces, the authors propose a framework that maps joint action spaces to a Riemannian manifold, making the problem more tractable. Assuming the reward functions are linear in terms of the learned mappings, the authors demonstrate that a no-regret algorithm can effectively learn optimal policies for both players. Experimental results suggest their approach outperforms baselines in certain game scenarios.

Strengths:

- The paper is well-written and clearly presented.
- Theoretical contributions and experimental results are sound.

Weaknesses:

- Motivation for the Riemannian manifold framework:
While the Riemannian manifold approach is mathematically elegant, the paper does not convincingly justify why such a complex framework is necessary compared to simpler alternatives, such as mappings in Euclidean space. This weakens the paper's overall motivation.
- Linearity assumption and efficiency:
The linearity assumption on the reward function is reasonable in theory, but its practicality diminishes when considering efficiency constraints, particularly when aiming for a low-dimensional representation. The authors do not adequately address how their framework balances this trade-off, leaving the motivation unclear.

Decision:
Reject. While the paper has clear strengths in presentation and theoretical grounding, the concerns about the motivation for the Riemannian manifold framework and the implications of the linearity assumption on efficiency prevent me from recommending acceptance.