One can then observe that $(\tilde{Z}_{j,t})_{j \in \mathbb{N}^*, t \in \mathbb{Z}}$ is still of the form (11) with either **C1** or **C'1** being satisfied. Using the integrability of order $2+\delta$ of the gradient, one can then use a law of large numbers and we get

$$ \lim_{k \to \infty} \frac{1}{n_k T_k} \sum_{j=1}^{n_k} \sum_{t=1}^{T_k} \tilde{Z}_{j,t} = \mathbb{E} (\tilde{Z}_{1,1}) = x'Vx \text{ a.s.} $$

▷ We then prove that if $\varepsilon > 0$,

$$ \sum_{t=1}^{+\infty} \mathbb{E} \left( W_{k,t}^2 \mathbf{1}_{|W_{k,t}| \ge \varepsilon} \right) \underset{k \to +\infty}{\longrightarrow} 0 $$

We have for any $k \in \mathbb{N}$

$$ \begin{aligned} \sum_{t=1}^{+\infty} \mathbb{E} \left( |W_{k,t}|^2 \mathbf{1}_{|W_{k,t}| \ge \varepsilon} \right) &= \sum_{t=1}^{T_k} \mathbb{E} \left( |V_{k,t}|^2 \mathbf{1}_{|V_{k,t}| \ge \varepsilon} \right) \\ &= \sum_{t=1}^{T_k} \frac{1}{T_k} \mathbb{E} \left( \left| \frac{1}{\sqrt{n_k}} \sum_{j=1}^{n_k} x' \nabla m_{\theta_0}(Z_{j,t}) \right|^2 \mathbf{1}_{\left\{ \left| \frac{1}{\sqrt{n_k}} \sum_{j=1}^{n_k} x' \nabla m_{\theta_0}(Z_{j,t}) \right| \ge \sqrt{T_k \varepsilon} \right\}} \right) \\ &= \mathbb{E} \left( |U_k|^2 \mathbf{1}_{|U_k|^2 \ge T_k \varepsilon^2} \right) \end{aligned} $$

denoting $U_k = \frac{1}{\sqrt{n_k}} \sum_{j=1}^{n_k} \nabla m_{\theta_0}(Z_{j,1})$. This last equality is indeed licit since the distribution of $(Z_{1,t}, \dots, Z_{N_k,t})$ is identical to the distribution of $(Z_{1,1}, \dots, Z_{N_k,1})$ by stationarity. It is then enough to show that the sequence $(U_k^2)_{k \ge 1}$ is uniformly integrable. Since $\mathbb{E}U_k^2 = x'Vx$, the result will follow from Theorem 3.6 Billingsley (2013) if we show that $U_k$ converges weakly to a random variable $U$ for which $\mathbb{E}(U^2) = x'Vx$. If **C1** is verified, one can use the central limit theorem for i.i.d. sequences to conclude. If now **C'1** is satisfied, the result is less obvious. To get the required convergence, we note that there exists a measurable mapping

$$ \tilde{g}: \mathbb{R}^k \times \mathbb{R}^{d_2} \times (\mathbb{R}^{d_6} \times \mathbb{R}^k)^N \times (\mathbb{R}^{d_4})^N \to \mathbb{R}^{d'} $$

such that, setting $\bar{\kappa}_{j,t} = (\varepsilon_{j,t}, \kappa_{j,t})$,

$$ Z_{j,1} = \tilde{g}(\varepsilon_{j,1}, V_j, (\bar{\kappa}_{j,1-s})_{s \ge 1}, (\eta_{1-s})_{s \ge 1}). $$

and, using the martingale difference property,

$$ \int x' \nabla m_{\theta_0} \circ \tilde{g}(u, V_j, (\bar{\kappa}_{j,1-s})_{s \ge 1}, (\eta_{1-s})_{s \ge 1}) \mathbb{P}_{\varepsilon_{1,1}}(du) = 0 $$

almost surely. From the independence assumptions, we deduce that $\mathbb{E}(x'\nabla m_{\theta_0}(Z_{j,1})|(\eta_s)_{s \ge 0}) = 0$ a.s. and that, conditionally on $(\eta_{1-s})_{s \ge 1}$, the variables $x'\nabla m_{\theta_0}(Z_{j,t})$, $j \ge 1$, are i.i.d., centered and the sequence $(U_k)_{k \ge 0}$ converges weakly to a zero-mean Gaussian random variables and variance

$$ V((\eta_{1-s})_{s \ge 1}) := \mathbb{E} \left( |x'\nabla m_{\theta_0}(Z_{1,1})|^2 | (\eta_{1-s})_{s \ge 1} \right). $$

We then deduce that the sequence $(U_k)_{k \ge 0}$ converges weakly to a mixture of Gaussian distributions, that is of the form $\sqrt{V((\eta_{1-s})_{s \ge 1})}W$ where $W$ follows a standard Gaussian distribution and is independent of the sequence $(\eta_{1-s})_{s \ge 1}$. We then deduce the uniform integrability of the sequence $(U_k)_{k \ge 0}$.

□
Let us denote $d_{\Theta} = \dim(\Theta)$. For any couple $(j,t) \in \mathbb{N}^* \times \mathbb{Z}$, and each index $k \in \{1, \dots, d_{\Theta}\}$, we apply the Taylor-Lagrange formula to $\frac{\partial m_{\theta}}{\partial \theta_k}(Z_{j,t})$ between $\hat{\theta}_{n,T}$ and $\theta_0$. There exists $\tilde{\theta}_k$ between $\hat{\theta}_{n,T}$ and $\theta_0$ such that

$$ \sum_{t=1}^{T} \sum_{j=1}^{n} \frac{\partial m_{\hat{\theta}_{n,T}}}{\partial \theta_k}(Z_{j,t}) = \sum_{t=1}^{T} \sum_{j=1}^{n} \frac{\partial m_{\theta_0}}{\partial \theta_k}(Z_{j,t}) + \sum_{t=1}^{T} \sum_{j=1}^{n} \sum_{l=1}^{d_{\Theta}} \frac{\partial^2 m_{\tilde{\theta}_k}}{\partial \theta_l \partial \theta_k}(Z_{j,t}) \cdot (\hat{\theta}_{n,T} - \theta_0)_l. $$

Therefore, if we denote $\nabla^2 m_{\tilde{\theta}}(Z_{j,t})$ the matrix

$$ \nabla^2 m_{\tilde{\theta}}(Z_{j,t}) \left( \frac{\partial^2 m_{\tilde{\theta}_k}}{\partial \theta_l \partial \theta_k}(Z_{j,t}) \right)_{1 \le l, k \le p}, $$

we get the formula

$$ \frac{1}{nT} \sum_{t=1}^{T} \sum_{j=1}^{n} \nabla m_{\hat{\theta}_{n,T}}(Z_{j,t}) = \frac{1}{nT} \sum_{t=1}^{T} \sum_{j=1}^{n} \nabla m_{\theta_0}(Z_{j,t}) + \frac{1}{nT} \sum_{t=1}^{T} \sum_{j=1}^{n} \nabla^2 m_{\tilde{\theta}}(Z_{j,t}) \cdot (\hat{\theta}_{n,T} - \theta_0). $$

This leads to the equation

$$ \sqrt{nT} \left( \mathbb{E} \left( \nabla^2 m_{\theta_0}(Z_{1,1}) + d_{n,T} \right) \cdot (\hat{\theta}_{n,T} - \theta_0) \right) = \frac{1}{\sqrt{nT}} \underbrace{\sum_{t=1}^{T} \sum_{j=1}^{n} \left\{ \nabla m_{\hat{\theta}_{n,T}}(Z_{j,t}) - \nabla m_{\theta_0}(Z_{j,t}) \right\}}_{G_{n,T}}, \quad (47) $$

where

$$ d_{n,T} = \frac{1}{nT} \sum_{t=1}^{T} \sum_{j=1}^{n} \nabla^2 m_{\tilde{\theta}}(Z_{j,t}) - \mathbb{E} \left( \nabla^2 m_{\theta_0}(Z_{1,1}) \right). $$

▷ We first prove that $d_{n,T} \xrightarrow{\mathbb{P}} 0$ when $n, T \to \infty$. We have indeed

$$ d_{n,T} = \frac{1}{nT} \sum_{t=1}^{T} \sum_{j=1}^{n} \left\{ \nabla^2 m_{\tilde{\theta}}(Z_{j,t}) - \mathbb{E} \left( \nabla^2 m_{\tilde{\theta}}(Z_{1,1}) \right) \right\} + \mathbb{E} \left( \nabla^2 m_{\tilde{\theta}}(Z_{1,1}) - \nabla^2 m_{\theta_0}(Z_{1,1}) \right). $$

Using Assumptions **P5** and **P7**, it is possible to apply Lemma 7.19, and we obtain

$$ \sup_{\theta \in \Theta} \left| \frac{1}{nT} \sum_{t=1}^{T} \sum_{j=1}^{n} \nabla^2 m_{\theta}(Z_{j,t}) - \mathbb{E} \left( \nabla^2 m_{\theta}(Z_{1,1}) \right) \right| \underset{n,T \to \infty}{\longrightarrow} 0 \quad \text{a.s.} $$

Thus

$$ \frac{1}{nT} \sum_{t=1}^{T} \sum_{j=1}^{n} \left\{ \nabla^2 m_{\tilde{\theta}}(Z_{j,t}) - \mathbb{E} \left( \nabla^2 m_{\tilde{\theta}}(Z_{1,1}) \right) \right\} \underset{n,T \to +\infty}{\longrightarrow} 0 \quad \text{a.s.} $$

Furthermore, since $\hat{\theta}_{n,T} \underset{n,T \to \infty}{\longrightarrow} \theta_0$ almost surely, we also have $\tilde{\theta} \underset{n,T \to \infty}{\longrightarrow} \theta_0$ almost surely, and Assumption **P7** ensure that

$$ \mathbb{E} \left( \nabla^2 m_{\tilde{\theta}}(Z_{1,1}) - \nabla^2 m_{\theta_0}(Z_{1,1}) \right) \underset{n,T \to \infty}{\longrightarrow} 0 $$

by Lebesgue's dominated convergence theorem. Here, we actually prove this latter convergence for any increasing sequences of integers $(n_k)_{k \in \mathbb{N}}$ and $(T_k)_{k \in \mathbb{N}}$. Hence, $d_{n,T} \underset{n,T \to \infty}{\longrightarrow} 0$ almost surely, and in particular $d_{n,T} = o_{\mathbb{P}}(1)$ when $n, T \to +\infty$.
▷ We then show that $G_{n,T} \xrightarrow{\mathcal{L}} \mathcal{N}(0, \text{Var}(\nabla m_{\theta_0}(Z_{1,1})))$ when $n, T \to \infty$.

Since $\hat{\theta}_{n,T}$ is a minimizer of $\theta \mapsto \sum_{t=1}^T \sum_{j=1}^n m_{\theta}(Z_{j,t})$, and since $\hat{\theta}_{n,T}$ converges almost surely to $\theta_0$ when $n, T \to \infty$, we almost surely have $\hat{\theta}_{n,T} \in \mathring{\Theta}$ for $n$ and $T$ large enough, and thus

$$ \sum_{t=1}^{T} \sum_{j=1}^{n} \nabla m_{\hat{\theta}}(Z_{j,t}) = 0, $$

hence

$$ \frac{1}{\sqrt{nT}} \sum_{t=1}^{T} \sum_{j=1}^{n} \nabla m_{\hat{\theta}_{N,T}}(Z_{j,t}) \xrightarrow{n,T \to \infty} 0 \quad \text{a.s.} \qquad (48) $$

Furthermore, we have by Lemma 7.21

$$ \frac{1}{\sqrt{nT}} \sum_{t=1}^{T} \sum_{j=1}^{n} \nabla m_{\theta_0}(Z_{j,t}) \xrightarrow{\mathcal{L}} \mathcal{N}(0, \text{Var}(\nabla m_{\theta_0}(Z_{1,1}))) $$

when $n, T \to \infty$. We thus obtain by Slutsky's lemma

$$ G_{n,T} \xrightarrow{\mathcal{L}} \mathcal{N}(0, \text{Var}(\nabla m_{\theta_0}(Z_{1,1}))) $$

when $n, T \to \infty$. Note that once again, this latter convergence is obtained for all increasing sub sequences of integers $(N_k)_{k \in \mathbb{N}}$ and $(T_k)_{k \in \mathbb{N}}$.

▷ Let us finally show that $\sqrt{nT}(\hat{\theta} - \theta_0) = O_P(1)$ when $n, T \to \infty$ and take any $\varepsilon > 0$.

By continuity of the determinant, there exists $\eta > 0$ such that for any matrix $x$ satisfying $\|x\| \le \eta$, $\mathbb{E}(\nabla^2 m_{\theta_0}(Z_{1,1})) + x$ is invertible. Let us denote

$$ S_{\eta} := \sup_{\|x\| \le \eta} \| (\mathbb{E}(\nabla^2 m_{\theta_0}(Z_{1,1})) + x)^{-1} \|, $$

which is finite by continuity of the inverse. We have for any $M > 0$ and $n, T \in \mathbb{N}^*$

$$ \mathbb{P} \left( \sqrt{nT} \| \hat{\theta}_{n,T} - \theta_0 \| > M \right) = \underbrace{\mathbb{P} \left( \sqrt{nT} \| \hat{\theta}_{n,T} - \theta_0 \| > M, \| d_{n,T} \| \le \eta \right)}_{A_{n,T}(M)} + \underbrace{\mathbb{P} \left( \sqrt{nT} \| \hat{\theta}_{n,T} - \theta_0 \| > M, \| d_{n,T} \| > \eta \right)}_{B_{n,T}(M)}. $$

On the one hand

$$ A_{n,T}(M) \le \mathbb{P} (S_{\eta} \times \|G_{n,T}\| > M, \|d_{n,T}\| \le \eta) \le \mathbb{P} \left( \|G_{n,T}\| > \frac{M}{S_{\eta}} \right). $$

Indeed, if $d_{n,T} \le \eta$, we have

$$ \begin{align*} \sqrt{nT} \| \hat{\theta}_{n,T} - \theta_0 \| > M &\iff \| (\mathbb{E}(\nabla^2 m_{\theta_0}(Z_{1,1})) + d_{n,T})^{-1} \cdot G_{n,T} \| > M \\ &\iff \| (\mathbb{E}(\nabla^2 m_{\theta_0}(Z_{1,1})) + d_{n,T})^{-1} \| \times \| G_{n,T} \| > M \\ &\iff S_{\eta} \times \| G_{n,T} \| > M. \end{align*} $$

Since $G_{n,T}$ converges in distribution, $G_{n,T} = O_P(1)$ and there exists $M_0 > 0$ and $n_A, T_A \in \mathbb{N}^*$ such that for all $n \ge n_A$ and all $T \ge T_A$

$$ A_{n,T}(M_0) \le \mathbb{P} \left( G_{n,T} \ge \frac{M_0}{S_{\eta}} \right) < \frac{\varepsilon}{2}. $$
On the other hand, $d_{n,T} = o_P(1)$, thus there exists $n_B, T_B \in \mathbb{N}^*$ such that for all $n \ge n_B$ and $T \ge T_B$

$$B_{n,T}(M_0) \le \mathbb{P}(\|d_{n,T}\| > \eta) \le \frac{\epsilon}{2}.$$

Hence, for $n \ge \max(n_A, n_B)$ and $T \ge \max(T_A, T_B)$ we have

$$\mathbb{P}\left(\sqrt{nT}\|\hat{\theta}_{n,T} - \theta_0\| > M\right) < \varepsilon,$$

i.e. $\sqrt{nT}(\hat{\theta}_{n,T} - \theta_0) = o_P(1)$ when $n, T \to \infty$.

We deduce from this last point that $d_{n,T} \cdot \sqrt{nT}(\hat{\theta}_{n,T} - \theta_0) = o_P(1)$ when $n, T \to +\infty$. Thus, from equations (47) and (48), we have

$$\mathbb{E}(\nabla^2 m_{\theta_0}(Z_{1,1})) \cdot \sqrt{nT}(\hat{\theta}_{n,T} - \theta_0) = -\frac{1}{\sqrt{nT}} \sum_{t=1}^{T} \sum_{j=1}^{n} \nabla m_{\theta_0}(Z_{j,t}) + o_P(1)$$

when $n, T \to \infty$.

The asymptotic normality then follows from Lemma 7.21 and Slutsky's lemma.
References

Ashford, J., & Sowden, R. (1970). Multi-variate probit analysis. *Biometrics*, 535–546.

Banys, P., Davydov, Y., & Paulauskas, V. (2010). Remarks on the slln for linear random fields. *Statistics & probability letters*, 80(5-6), 489–496.

Billingsley, P. (2013). *Convergence of probability measures*. John Wiley & Sons.

Candelon, B., Dumitrescu, E.-I., Hurlin, C., & Palm, F. C. (2013). Multivariate dynamic probit models: An application to financial crises mutation. In *Var models in macroeconomics-new developments and applications: Essays in honor of christopher a. sims* (pp. 395–427). Emerald Group Publishing Limited.

CEFAS. (2024). *Fish stomach records*. Retrieved June 3, 2024, from https://www.cefas.co.uk/data-and-publications/fish-stomach-records/

Chaubert, F., Mortier, F., & Saint André, L. (2008). Multivariate dynamic model for ordinal outcomes. *Journal of Multivariate Analysis*, 99(8), 1717–1732.

Chib, S., & Greenberg, E. (1998). Analysis of multivariate probit models. *Biometrika*, 85(2), 347–361.

De Jong, R. M., & Woutersen, T. (2011). Dynamic time series binary choice. *Econometric Theory*, 27(4), 673–702.

Debaly, Z. M., Neumann, M. H., & Truquet, L. (2023). Mixing properties of nonstationary multivariate count processes. *arXiv preprint arXiv:2311.10692*.

Debaly, Z. M., & Truquet, L. (2021). Iterations of dependent random maps and exogeneity in nonlinear dynamics. *Econometric Theory*, 37(6), 1135–1172.

Debaly, Z.-M., & Truquet, L. (2023). Multivariate time series models for mixed data. *Bernoulli*, 29(1), 669–695.

de Leon, A. R. (2005). Pairwise likelihood approach to grouped continuous model and its extension. *Statistics & probability letters*, 75(1), 49–57.

DiCiccio, T. J., & Efron, B. (1996). Bootstrap confidence intervals. *Statistical science*, 11(3), 189–228.

Fernández-Val, I., & Weidner, M. (2016). Individual and time effects in nonlinear panel models with large n, t. *Journal of Econometrics*, 192(1), 291–312.

Giap, D. X., & Van Quang, N. (2016). Multidimensional and multivalued ergodic theorems for measure-preserving transformations. *Set-Valued and Variational Analysis*, 24, 637–658.

Greene, W. (2009). Discrete choice modeling. In *Palgrave handbook of econometrics: Volume 2: Applied econometrics* (pp. 473–556). Springer.

Guanche, Y., Minguez, R., & Méndez, F. J. (2014). Autoregressive logistic regression applied to atmospheric circulation patterns. *Climate dynamics*, 42, 537–552.

Hahn, J., & Kuersteiner, G. (2011). Bias reduction for dynamic nonlinear panel models with fixed effects. *Econometric Theory*, 27(6), 1152–1191.

Hajivassiliou, V. A., & Ruud, P. A. (1994). Classical estimation methods for ldv models using simulation. *Handbook of econometrics*, 4, 2383–2441.

Hsiao, C. (2022). *Analysis of panel data*. Cambridge university press.

ICES. (2024). *North sea international bottom trawl survey*. Retrieved May 29, 2024, from https://datras.ices.dk/Data_products/Download/Download_Data_public.aspx

Jentsch, C., & Reichmann, L. (2022). Generalized binary vector autoregressive processes. *Journal of Time Series Analysis*, 43(2), 285–311.

Kenne Pagui, E. C., & Canale, A. (2016). Pairwise likelihood inference for multivariate ordinal responses with applications to customer satisfaction. *Applied Stochastic Models in Business and Industry*, 32(2), 273–282.

Klesov, O., & Molchanov, I. (2017). Moment conditions in strong laws of large numbers for multiple sums and random measures. *Statistics & Probability Letters*, 131, 56–63.
Krengel, U. (2011). *Ergodic theorems* (Vol. 6). Walter de Gruyter.

Liesenfeld, R., & Richard, J.-F. (2010). Efficient estimation of probit models with correlated errors. *Journal of Econometrics*, **156**(2), 367–376.

Manner, H., Türk, D., & Eichler, M. (2016). Modeling and forecasting multivariate electricity price spikes. *Energy Economics*, **60**, 255–265.

Manski, C. F. (1975). Maximum score estimation of the stochastic utility model of choice. *Journal of econometrics*, **3**(3), 205–228.

Masarotto, G., & Varin, C. (2012). Gaussian copula marginal regression. *Electronic Journal of Statistics*, **6** (none), 1517–1549. https://doi.org/10.1214/12-EJS721

Müller, G., & Czado, C. (2005). An autoregressive ordered probit model with application to high-frequency financial data. *Journal of Computational and Graphical Statistics*, **14**(2), 320–338.

Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. *The computer journal*, **7**(4), 308–313.

Nguyen, X. X., & Zessin, H. (1979). Ergodic theorems for spatial processes. *Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete*, **48**(2), 133–158.

Park, J. Y., & Phillips, P. C. (2000). Nonstationary binary choice. *Econometrica*, **68**(5), 1249–1280.

Pfanzagl, J. (1969). On the measurability and consistency of minimum contrast estimates. *Metrika*, **14**(1), 249–272.

Poggiato, G., Münkemüller, T., Bystrova, D., Arbel, J., Clark, J. S., & Thuiller, W. (2021). On the interpretations of joint modeling in community ecology. *Trends in Ecology & Evolution*, **36**(5), 391–401.

Powell, M. J. (1994). *A direct search optimization method that models the objective and constraint functions by linear interpolation*. Springer.

Propp, J. G., & Wilson, D. B. (1996). Exact sampling with coupled markov chains and applications to statistical mechanics. *Random Structures & Algorithms*, **9**(1-2), 223–252.

Samorodnitsky, G., et al. (2016). *Stochastic processes and long range dependence* (Vol. 26). Springer.

Sebastian-Gonzalez, E., Sanchez-Zapata, J. A., Botella, F., & Ovaskainen, O. (2010). Testing the heterospecific attraction hypothesis with time-series data on species co-occurrence. *Proceedings of the Royal Society B: Biological Sciences*, **277**(1696), 2983–2990.

Straumann, D. (2006). *Estimation in conditionally heteroscedastic time series models* (Vol. 181). Springer Science & Business Media.

Talhouk, A., Doucet, A., & Murphy, K. (2012). Efficient bayesian inference for multivariate probit models with sparse inverse correlation matrices. *Journal of Computational and Graphical Statistics*, **21**(3), 739–757.

Ting, B., Wright, F., & Zhou, Y.-H. (2022). Fast multivariate probit estimation via a two-stage composite likelihood. *Statistics in Biosciences*, **14**(3), 533–549.

Truquet, L. (2023). Strong mixing properties of discrete-valued time series with exogenous covariates. *Stochastic Processes and their Applications*, **160**, 294–317.

Tuzcuoglu, K. (2023). Composite likelihood estimation of an autoregressive panel ordered probit model with random effects. *Journal of Business & Economic Statistics*, **41**(2), 593–607.

Varin, C., Reid, N., & Firth, D. (2011). An overview of composite likelihood methods. *Statistica Sinica*, **5–42**.

Wilkinson, D. P., Golding, N., Guillera-Arroita, G., Tingley, R., & McCarthy, M. A. (2019). A comparison of joint species distribution models for presence-absence data. *Methods in Ecology and Evolution*, **10**(2), 198–211.

Young, G., Valdez, E. A., & Kohn, R. (2009). Multivariate probit models for conditional claim-types. *Insurance: Mathematics and Economics*, **44**(2), 214–228.
Zhao, Y., & Joe, H. (2005). Composite likelihood estimation in multivariate data analysis.
*Canadian Journal of Statistics*, **33**(3), 335–356.