# MISTA: compact mutil-identity structure-aware
tensorized avatars

Oumaima Badi, Xiaoran Jiang, Luce Morin, Mårten Sjöström

► To cite this version:

Oumaima Badi, Xiaoran Jiang, Luce Morin, Mårten Sjöström. MISTA: compact mutil-identity structure-aware tensorized avatars. 2026 IEEE International Conference on Acoustics, Speech, and Signal Processing, May 2026, Bacelone, Spain. hal-05481668

HAL Id: hal-05481668

https://hal.science/hal-05481668v1

Submitted on 28 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# MISTA: COMPACT MULTI-IDENTITY STRUCTURE-AWARE TENSORIZED AVATARS

Oumaima Badi<fnref n="1" />, Xiaoran Jiang<fnref n="1" />, Luce Morin<fnref n="1" />, Mårten Sjöström²

¹Univ. Rennes, INSA Rennes, CNRS, IETR, UMR 6164, Rennes, France

²Mid Sweden University, Sundsvall, Sweden

## ABSTRACT

This paper addresses the challenge of learning a shared and compact representation of multi-identity human avatars from monocular videos. Most existing approaches rely on per-identity optimization, which lacks generalizability and is costly in both time and storage. We propose MISTA<fnref n="1" />, a method that applies Tensor Train (TT) decomposition to approximate the global tensor of 3D Gaussian (3DG) parameters for multiple avatars in canonical pose. Before decomposition, the parameters are reordered and reshaped in a structure-aware manner to better exploit spatial correlations. Experiments demonstrate that MISTA achieves reconstruction and animation quality comparable to the state-of-the-art method MIGS while using only 6.5% of its parameters, making it particularly suitable for deployment on resource-constrained devices.

**Index Terms—** Human avatars, 3D Gaussian splatting, Low-rank tensor decomposition, Compact representation.

## 1. INTRODUCTION AND RELATED WORKS

Learning photorealistic and animatable human avatars is becoming increasingly important across various domains, including sports, cinema and visual effects, and AR/VR, enabling free-viewpoint replays, digital doubles, and real-time telepresence. Unlike approaches relying on multi-view video capture systems, monocular video-based methods are particularly relevant for real-world use, as they avoid complex acquisition setups and instead operate on easily accessible content, such as videos captured with a single mobile phone.

Modeling humans from monocular video is fundamentally ill-posed, as depth and occluded geometry cannot be recovered from a single view. This challenge is often mitigated with kinematic priors such as SMPL [1], which provides a parametric body template for pose representation and canonicalization.

**Overfitted per-identity models:** Most existing approaches overfit a model, either implicit Neural Radiance Fields (NeRFs) [2, 3, 4, 5] or explicit 3D Gaussian Splatting (3DGS) [6, 7, 8, 9], to monocular videos of a single identity.

Model parameters are optimized by animating the learned 3D avatar with SMPL priors and reprojecting to the 2D image space to match the input frames.

Among the pioneering works, HumanNeRF [3] represents humans as a canonical radiance field with pose-conditioned deformations, but it suffers from slow training and rendering because of NeRF's dense per-ray sampling. InstantAvatar [5], inspired by Fast-SNARF [10], alleviates this limitation through empty space skipping, which accelerates training and inference but compromises rendering fidelity for speed. Moving to explicit representations, 3DGS-Avatar [7] initializes Gaussians from SMPL and applies blend skinning with lightweight MLPs to enable real-time rendering. GauHuman [9] focuses on efficiency by directly optimizing Gaussians from SMPL-aligned depth maps, which allows faster reconstruction while maintaining reasonable quality.

**Generalization across identities:** However, the overfitting nature of these methods limit their scalability. Each new subject requires optimizing an independent and complete set of parameters, causing the training cost and memory requirements to grow linearly with the number of subjects. Research has taken two main directions. The first relies on prior models that learn identity-agnostic representations. For instance, Vid2AvatarPro [11] learns a universal prior from large-scale multi-view, multi-identity captures, using canonical mappings to handle clothing and adapting to monocular videos through finetuning. Parametric Gaussian Human Models [12] introduce a compact UV-aligned identity map and disentangled decoders for pose and view-dependent details. While these methods reduce per-identity training, their model size and personalization cost remain substantial. Another approach focuses on shared low-rank representations, assuming that multi-identity avatars are correlated and can be projected to a low-dimensional manifold. MIGS [13] applies Canonical Polyadic (CP) decomposition [14] to the Gaussian parameters tensor, compressing redundancy and enabling a single model for multiple avatars. But a single global rank is imposed across modes and the Gaussian axis is treated as flat, which restricts flexibility across dimensions.

**Contributions:** In this paper, we aim to simultaneously enhance the compactness and generalizability of multi-identity 3DGS models, without sacrificing reconstruction quality or requiring large-scale training data. To this end,

<footnote n="1">¹Code will be available at https://github.com/OumaimaBadi/MISTA</footnote>
we propose a method for compact Multi-Identity Structure-Aware Tensorized Avatars (MISTA), the first structure-aware Tensor Train (TT) factorization [15] for multi-identity 3DGS avatars. Unlike CP, TT allows different ranks across modes, offering more flexibility. We spatially reorder Gaussians and reshape them into balanced local components before applying TT. Experiments on a multi-identity video dataset [16] show that our approach uses only 6.5% of the parameters of the reference method MIGS [13], while maintaining comparable visual quality.

## 2. PROPOSED METHOD

### 2.1. Overview

Our approach targets compact and generalizable multi-identity avatar representations learned from monocular videos, with model compactness and learning efficiency as primary requirements for practical deployment. We follow the shared low-rank representation paradigm, avoiding prior-based methods that depend on large-scale pretraining.

Multi-identity Gaussian avatars can be modeled as a dense tensor $X \in \mathbb{R}^{I \times G \times M}$, where $I$ is the number of identities, $G$ the number of Gaussians per identity, and $M$ the parameters per Gaussian. Directly handling $X$ becomes prohibitive as $I$ increases. Under a low-rank assumption, MIGS [13] alleviates this via CP decomposition, but its fixed global rank limits flexibility. Instead, MISTA adopts a structure-aware TT decomposition with ranks that adapt across dimensions.

Training starts by initializing Gaussian positions from a canonical SMPL mesh using a single identity, yielding $X$ of size $1 \times G \times M$. Since Gaussians are randomly ordered, we reorder them using a 3D Hilbert space-filling curve, leveraging the assumption that spatially nearby Gaussians tend to share similar opacity, covariance, and spherical harmonic coefficients. The reordered tensor is then reshaped into a higher-order tensor $\hat{X}$ by splitting the Gaussian mode into multiple smaller sub-modes, as TT benefits from balanced high-order tensors [15]. We apply TT decomposition to $\hat{X}$ to obtain compact cores $C_k$ with ranks $r_k$, then expand the identity core from one identity to $I$ identities. The canonical avatars are recovered from the cores and animated following the 3DGS-Avatar [7] pipeline using three MLPs: two for non-rigid and rigid transformations and one for color/appearance. The resulting Gaussians are rasterized with alpha blending to render the final images.

Note that TT decomposition is performed only once at the beginning of training; the trainable parameters are the TT cores $C_k$ and the MLP weights. An overview of MISTA is shown in Fig. 1.

### 2.2. Incorporating Tensor Train Decomposition

Our goal is to obtain a compact tensor representation that is shared between multi-identities' avatars. CP [14] achieves

compact representations but is rigid due to its single global rank, while Tucker [17] allows flexible ranks but introduces a large core that grows quickly with dimensionality. TT [15] strikes a balance by factorizing the tensor into a sequence of small cores, achieving both compactness and adaptability, which makes it well suited for our setting. Given a tensor $X \in \mathbb{R}^{n_1 \times n_2 \times \cdots \times n_d}$, each entry can be approximated as

$$X(i_1, i_2, \dots, i_d) \approx C_1(i_1) C_2(i_2) \cdots C_d(i_d), \quad (1)$$

where each TT core is a three-dimensional tensor $C_k \in \mathbb{R}^{r_{k-1} \times n_k \times r_k}$, and $C_k(i_k) \in \mathbb{R}^{r_{k-1} \times r_k}$ denotes its slice corresponding to index $i_k$. Here $n_k$ is the size of mode $k$, and $r_k$ is the TT-rank, with boundary ranks $r_0 = r_d = 1$.

The number of parameters is $\sum_{k=1}^{d} r_{k-1} n_k r_k$, which scales with the TT-ranks rather than with the full tensor size $\prod_{k=1}^{d} n_k$, thereby providing a compact representation.

### 2.3. Spatial Reshaping of the Gaussian Mode

Since TT decomposition is more effective on high-order tensors with balanced mode sizes, we reshape the Gaussian dimension of $X$ into multiple submodes. Specifically, the number of Gaussians $G$ is factorized as $G = G_1 G_2 \cdots G_L$, where $L$ is the number of Gaussian submodes. This yields the reshaped tensor $\hat{X} \in \mathbb{R}^{I \times G_1 \times G_2 \times \cdots \times G_L \times M}$.

The purpose of this reshaping is to expose local spatial correlations, which TT can exploit more effectively than with a single long dimension $G$. To this end, we restrict to factorizations $(G_1, \dots, G_L)$ that are balanced, i.e., with similar submode sizes, to avoid creating an extremely elongated dimension. For each candidate factorization, we also evaluate its permutations, since the ordering of submodes directly affects neighborhood structure in the reshaped tensor. We formalize this with an adjacency cost $A$:

$$A(G_1, \dots, G_L) = \sum_{i=1}^{G} \sum_{j \in \mathcal{N}(i)} \|p_i - p_j\|_2^2, \quad (2)$$

where $p_i \in \mathbb{R}^3$ is the 3D position of Gaussian $i$, and $\mathcal{N}(i)$ denotes its direct neighbors along each reshape dimension. The factorization and permutation that minimize $A$ are chosen, as they best preserve spatial proximity of Gaussians in $\hat{X}$, enabling TT to capture correlations more effectively.

### 2.4. Structure-Aware Gaussian Re-ordering

In the original tensor $X$, Gaussian parameters follow the random initialization order, so adjacent indices do not reflect spatial proximity. Since TT decomposition benefits from local correlations [18], especially after reshaping into higher order, we impose a geometric ordering using a 3D Hilbert space-filling curve [19, 20]. This curve maps each position $(x, y, z)$, normalized to $[0, 1]^3$, to a one-dimensional Hilbert index $h$ by
<figure><img src="image_2.png" /><figcaption>**Fig. 2.** Qualitative results on ZJU-MoCap: unseen view synthesis and unseen pose animation on three subjects seen during training. Number of parameters is 5.01M for MIGS, and 326k for MISTA.</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>**Fig. 3.** Qualitative results on ZJU-MoCap for unseen identity: fine-tuning on a subject not included in the training set. Number of parameters is 5.01M for MIGS, and 326k for MISTA.</figcaption></figure>

Visual comparisons between MISTA and MIGS ($r = 100$) are shown in Figure 2 and Figure 3. The two models achieve similar visual quality for new view and new pose rendering (Figure 2), whereas for unseen identity rendering (Figure 3), MIGS performs slightly better. Note once again that our MISTA model has only 6.5% parameters compared to MIGS ($r = 100$). More configurations are compared in Figure 4 in terms of model size and LPIPS quality.

### 3.3. Ablation study

To better understand the design choices of MISTA, we first examine the effect of tensor reshaping. Figure 4 compares three variants of MISTA (MISTA-4D, MISTA-5D, and MISTA-6D) against 2 MIGS models ($r = 10$ and $r = 100$). Among them, MISTA-5D with a balanced 5D reshape (8, 50, 40, 25, 43) achieves the best quality vs. model size tradeoff. By contrast, in MISTA-4D configuration, the two Gaussian dimensions (250 x 200) are too much unbal-

<figure><img src="image_4.png" /><figcaption>**Fig. 4.** Comparison between MISTA reshapes (4D, 5D, 6D) and MIGS ($r=10, r=100$) in terms of LPIPS ×10³ versus parameter count (log scale). Each point is annotated with its tensor shape ($n_k$), TT-ranks ($r_k$), and number of parameters.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>**Fig. 5.** Visual comparison of MISTA models with and without Hilbert reordering on subject 386, with the corresponding LPIPS*.</figcaption></figure>

anced compared to the first and the last dimension, which weakens the trade-off. MISTA-6D introduces an additional core and more TT ranks as hyperparameters to adjust, with limited benefit however.

Figure 5 compares the visual quality between the proposed MISTA and that without Hilbert Gaussian re-ordering. The quality when removing Hilbert re-ordering degrades significantly (LPIPS reported under the images on subject 386). Measured over the entire ZJU-MoCap dataset, removing Hilbert ordering degrades LPIPS from 36.50 to 47.21.

## 4. CONCLUSION

This work presents a structure-aware Tensor Train decomposition for compact multi-identity Gaussian avatars, achieving rendering quality comparable to MIGS with only 6.5% of its parameters. Results demonstrate robustness across unseen views, poses, and unseen identities. A promising direction is to couple TT rank selection with rendering losses, enabling adaptive model size configuration and rank-aware training that balances compactness with fidelity.

## 5. ACKNOWLEDGMENT

This study is supported by the ANR within the framework of the PIA EUR DIGISPORT project (ANR-18-EURE-0022).
## 6. REFERENCES

[1] M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black, “Smpl: A skinned multi-person linear model,” *ACM Trans. Graph.*, vol. 34, no. 6, pp. 248:1–248:16, 2015.

[2] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” in *ECCV*, 2020.

[3] C.-Y. Weng, B. Curless, P. P. Srinivasan, J. T. Barron, and I. Kemelmacher-Shlizerman, “Humannerf: Free-viewpoint rendering of moving people from monocular video,” in *CVPR*, 2022, pp. 16210–16220.

[4] Z. Yu, W. Cheng, X. Liu, W. Wu, and K.-Y. Lin, “Mono-human: Animatable human neural field from monocular video,” in *CVPR*, 2023, pp. 16943–16953.

[5] T. Jiang, X. Chen, J. Song, and O. Hilliges, “Instanthatar: Learning avatars from monocular video in 60 seconds,” in *CVPR*, 2023.

[6] B. Kerbl, G. Kopanas, T. Leimkuehler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering,” *ACM Trans. Graph.*, vol. 42, no. 4, pp. 139:1–139:14, 2023.

[7] Z. Qian, S. Wang, M. Mihajlovic, A. Geiger, and S. Tang, “3dgs- avatar: Animatable avatars via deformable 3d gaussian splatting,” in *CVPR*, 2024.

[8] L. Hu, H. Zhang, Y. Zhang, B. Zhou, B. Liu, S. Zhang, and L. Nie, “Gaussian avatar: Towards realistic human avatar modeling from a single video via animatable 3d gaussians,” in *CVPR*, 2024.

[9] S. Hu, T. Hu, and Z. Liu, “Gauhuman: Articulated gaussian splatting from monocular human videos,” in *CVPR*, 2024, pp. 20418–20431.

[10] X. Chen, T. Jiang, J. Song, M. Rietmann, A. Geiger, M. J. Black, and O. Hilliges, “Fast-snarf: A fast deformer for articulated neural fields,” *IEEE Trans. Pattern Anal. Mach. Intell.*, vol. 45, no. 10, pp. 11796–11809, 2023.

[11] C. Guo, J. Li, Y. Kant, Y. Sheikh, S. Saito, and C. Cao, “Vid2avatar-pro: Authentic avatar from videos in the wild via universal prior,” in *CVPR*, 2025.

[12] C. Peng, J. Sun, Y. Chen, Z. Su, Z. Su, and Y. Liu, “Parametric gaussian human model: Generalizable prior for efficient and realistic human avatar modeling,” *arXiv preprint arXiv:2506.06645*, 2025.

[13] A. Chatziagapi, G. G. Chrysos, and D. Samaras, “Migs: Multi-identity gaussian splatting via tensor decomposition,” in *ECCV*, 2024.

[14] T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” *SIAM Review*, vol. 51, no. 3, pp. 455–500, 2009.

[15] I. V. Oseledets, “Tensor-train decomposition,” *SIAM J. Sci. Comput.*, vol. 33, no. 5, pp. 2295–2317, 2011.

[16] S. Peng, Y. Zhang, Y. Xu, Q. Wang, Q. Shuai, H. Bao, and X. Zhou, “Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans,” in *CVPR*, 2021, pp. 9054–9063.

[17] I. V. Oseledets, D. V. Savostianov, and E. E. Tyrtyshnikov, “Tucker dimensionality reduction of three-dimensional arrays in linear time,” *SIAM J. Matrix Anal. Appl.*, vol. 30, no. 3, pp. 939–956, 2008.

[18] Y. Panagakis, J. Kossaifi, G. G. Chrysos, J. Oldfield, M. A. Nicolaou, A. Anandkumar, and S. Zafeiriou, “Tensor methods in computer vision and deep learning,” *Proc. IEEE*, vol. 109, no. 5, pp. 863–890, 2021.

[19] D. Hilbert, “Über die stetige abbildung einer linie auf ein flächenstück,” *Math. Ann.*, vol. 38, no. 3, pp. 459–460, 1891.

[20] P. Franco, G. Nguyen, and R. Mullot, “Alternative patterns of the multidimensional hilbert curve,” *Multimedia Tools Appl.*, vol. 77, pp. 8419–8440, 2018.

[21] G. M. Morton, “A computer oriented geodetic data base and a new technique in file sequencing,” *Tech. Rep.*, IBM, 1966.

[22] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” *arXiv preprint arXiv:1412.6980*, 2014.

[23] J. Kossaifi, Y. Panagakis, A. Anandkumar, and M. Pantic, “Tensorly: Tensor learning in python,” *J. Mach. Learn. Res.*, vol. 20, no. 26, pp. 1–6, 2019.

[24] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” *IEEE Trans. Image Process.*, vol. 13, no. 4, pp. 600–612, 2004.

[25] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in *CVPR*, 2018, pp. 586–595.