Curvature-Guided Task Synergy for Skeleton based Temporal Action Segmentation
Authors: Guozhang Li
Xinran Duan
Mei Wang
Lizhi Wang
Hua Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Temporal Action Segmentation
Skeleton-based Learning
Geometric Priors
Curvature Guidance
Task Synergy
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Fine-grained temporal action segmentation plays a vital role in comprehensivehuman behavior understanding, with skeleton-based approaches (STAS) gaining prominence for their privacy and robustness. A core challenge in STAS arises from the conflicting feature requirements of action classification (demanding temporal invariance) and boundary localization (requiring temporal sensitivity). Existing methods typically adopt decoupled pipelines, unfortunately overlooking the inherent semantic complementarity between these sub-tasks, leading to information silos that prevent beneficial cross-task synergies. To address this challenge, we propose CurvSeg, a novel approach that synergizes classification and localization within the STAS domain through a unique geometric curvature guidance mechanism. Our key innovation lies in exploiting curvature properties of well-learned classification representations on skeleton sequences. Specifically, we observe that high curvature within action segments and low curvature at transitions effectively serve as geometric priors for precise boundary detection. CurvSeg establishes a virtuous cycle: localization predictions, guided by these curvature
signals, in turn dynamically refine the classification feature space to organize into a geometry conducive to clearer boundaries. To compute stable curvature signals from potentially noisy skeleton features, we further develop a dual-expert weighting mechanism within a Mixture of Experts framework, providing task-adaptive feature extraction. Comprehensive experiments demonstrate that CurvSeg signif-icantly enhances STAS performance across multiple benchmark datasets, achieving superior results and validating the power of geometric-guided task collaboration for this specific problem.

Published as a conference paper at ICLR 2026
CURVATURE-GUIDED TASK SYNERGY FOR SKELETON BASED TEMPORAL ACTION SEGMENTATION
Guozhang Li, Xinran Duan, Mei Wang Lizhi Wang, Hua Huang
School of Artificial Intelligence, Beijing Normal University, Beijing BeijingKey Laboratory of Aritifical Intelligence for Education Engineering Research Center of Intelligent Technology and Educational Application, Ministry of Education
ABSTRACT
Fine-graa Fine-grained temporal action segmentation plays vital role in comprehensive human behavior understanding, with skeleton-based approaches (STAS) gaining prominence for their privacy and robustness. A core challenge in STAS arises from the conflicting feature requirements of action classification (demanding tem- poral invariance) and boundary localization (requiring temporal sensitivity). Ex- isting methods typically adopt decoupled pipelines, unfortunately overlooking the inherent semantic complementarity between these sub-tasks, leading informa- tion silos that prevent beneficial tasss-task synergies. To address this challenge, we propose CurvSeg, novel approach that synergizes classification and local- ization within the STAS domain through a unique geometric curvature guidance mechanism. Our key innovation lies exploiting curvature properties of well- learned classification representations on skeleton sequences. Specifically, we ob- serve that high curvature within action segments and low curvature at transitions effectively serve as geometric priors for precise boundary detection. CurvSeg establishes a virtuous cycle: localization predictions, guided by these curvature signals, in turn dynamically refine the classification feature space to organize into a geometry conducive to clearer boundaries. To compute stable curvature signals from potentially noisy skeleton features, further develop a dual-expert weight- ing mechanism within a Mixture of Experts framework, providing task- adaptive feature extraction. Comprehensive experiments demonstrate that CurvSeg signif- icantly enhances STAS performance across multiple benchmark datasets, achiev- ing superior results and validating the power of coometricc-uidde task collabora- tion for this specific problem. CurvSeg
INTRODUCTION
Temporal Action Segmentation (TAS), which precisely assigns action labels to every frame in untrimmed videos, is fundamental task for fine-grained human behavior understanding. While video-based methods (Farha Gall 2019 Li al. 2023b Yi et al. 2021 Du et al., 2022 Zhao & Song. 2022 Ren et al. 2025 Liu et al. 2023b Li et al. 2023a) have achieved great progress, they often face challenges in scenarios demanding high privacy and robustness to varying appearances or visual noise. Skeleton-based TAS (STAS) thus emerges as vital alternative, modeling pure motion dynamics to inherently ensure privacy and decouple from visual confounders, making it crucial for sensitive domains like healthcare.
A central challenge in STAS is the fundamental tension between its two sub-tasks: action classifi- cation and boundary localization. Classification requires temporally invariant, abstract features to ensure consistent recognition within a segment. In contrast, localization demands temporally sen- sitive, ane-grrined features to pinpoint the exact moment of transition. The prevailing paradigm to resolve this conflict is task decoupling, which employs separate decoders for each task atop shared spatio-temporal encoder (e.g., GCNs and TCNs) (Filtjens et al. 2022 Li al. 2024a 2021a Ghosh et al. 2020 Li et al.) 2023d c Yan et al., 2018 Chai et al. 2024). as shown in Fig. a). However, we argue this popular strategy is a critical ver-simplification While the features may compete, the
Corresponding author: Mei Wang (wangmeil @ hhnueddu.c), Xinran Duan contributed equally this work.

---

Published as a conference paper at ICLR 2026
tasks themselves are highly complementary semantic level; knowing what action is occurring provides powerful priors for where its boundaries lie, and vice versa. By isolating the two, current methods create "information silos" that preclude beneficial cross-task synergies and artificially limit performance.
Recent refinements include decoupling spatio-temporal modeling mitigate feature over- smoothing (Li et al.] 2023d) and integrating language priors for enhanced representation learn- ing (Ji et al. 2024) but these improvements do not address the fundamental limitation of insufficient cross-task collaboration. To address this limitation, we introduce Curvature-Guidee Task Synergy (CurvSeg) for STAS, which restores the intrinsic synergy between decoupled tasks through two key innovations: curvature-based synergy and adaptive feature extraction, as shown in Fig. (b).
Our approach is inspired by a profound geo- metric insight, recently explored in representa- tion learning (Shin et al.) 2004)), as shown in
High Cury.
+
Fig. [c): in well-learned feature space, tra-
Cl.. Head
Loc. Head
Ch. Head
Loc: Head
jectories of a continuous data sequence (like skeleton frames) are spatially confined within their respective class clusters. This confine- ment naturally induces high curvature through- out an action segment as the feature path con- tinuously turns avoid crossing the cluster's s boundary. Conversely, the path straightens out during transitions between actions, resulting in low-curvature "valleys". Connequently we first exploit this geometric property for cross-task collaboration in STAS, establishing a synergis- tie loop between feature learning and tempo- ral localization. The curvature of classification features serves as a geometric prior to guide boundary detection, with curvature valleys indicating likely transition points. Reciprocally, lo- calization predictions supervise the classification feature space by penalizing low curvature within predicted action segments. This symbiotic relationship encourages the generation of more discrim- inative, compact clusters, which yield more precise geometric priors, creating a synergistic training strategy of improvement.
Infommation
Curvature hased Synergy
Wisingg Coos--135k
Representazion Space
Cls. Experts
Loc. Experts
(4)
Features
curv.
sequence
Spatio- Temporal Encoder
Spatio Temporal
Head
Head
Boundary
Prediction
(+)
(b)
(d)
Figure 1: (a) Existing STAS approaches. (b)
The proposed method. (c) Classification repre-
sentations create compact clusters, forcing high trajectory curvature within action segments. (d)
survature-guider task collaboration.
However, the effectiveness of this curvature-based synergy depends on the quality of the underlying features. Current approaches of using single, shared encoder output for both tasks creates com- promised representation, failing to optimally serve either classification or localization. To overcome this limitation, we introduce a dual-expert weighting mechanism within Mixture of Experts (MoE) framework that enables task-adaptive feature extraction from the shared encoder outputs. Specifi- cally, we deploy a separate expert module: the classification expert extracts comantic-oriented rep- resentations optimized for action recognition, while the localization expert focuses on fine-graines temporal details essential for boundary detection. This dual-expert mechanism operates across both temporal and spatial dimensions, enabling task-specific feature refinement that optimizes representa- tions for their respective objectives. By providing task-specific feature distillation rather than shared representations, our MoE framework ensures that each decoder head receives optimally tailored inputs, creating an ideal foundation for the curvature- based task collaboration strategy.
The contributions of this work are threefold: First, it proposes novel curvature- based task syn- ergy mechanism that exploits the geometric properties of feature sequences to establish effective collaboration between decoupled classification and localization sub-tasks. Second, it introduces dual-expert weighting mechanism within a MoE framework that provides task-adaptive feature ex- traction, with separate experts extracting semantic representations for classification and fine- -raingd temporal details for localization to improve the performance of synergy mechanism. Third, compre- hensive experiments demonstrate that our crvatuue-bbsed synergy mechanism effectively enhances action segmentation performance across multiple benchmark datasets.
2

---

Published as a conference paper at ICLR 2026
Expert-Driven Decoupling
CE loss
Predictions
BCE loss
Curvature-Guided Synergy
Concatenate
cb (
Boundary Prediction
mm11
Collaburation
M
Pre 200
+
Curvature-Guided Synergy
Routr
Curvature Signal Sequence
Gaussian Generator
Cls. Head
Loc. Head
(Ftmi mm-1
P Felaten
Split
Fst
Expert-Driven Decoupling
Fetet
Linformer
Temporal Spatio
Cl.. Experts
Linformer
Experts
O
MS GCN
Linformer
A
F
Router
GCN
Representations Space
MS Adjacency Matrix
Cls. Features
:dddition
X Multiplication
Ah; Adjacency Matrix
Fsyt Spatio-Temport Feature
Fim: Segment Feature
Gaussian Expert
Fats Cls. Feature
#: Feature Turning, Angle
Figure 2: Overview of CurvSeg. Our model processes video features through EDD to capture de- coupled classification and localization representations utilizing task specifciiiicc experts. Subsequently, CGS leverages geometric curvature principles to guide task collaboration, enhancing both action boundary detection and classification performance.
RELATED WORK
Temporal Action Segmentation. Video-based TAS methods primarily utilize RGB or optical flow
features for action understanding.
Early RNN-based approaches focused on sequential model-
ing (Ding & Xu 2017: Singh et al. 2016). followed by TCN- based methods to capture long-range range
dependencies (Lea et al.) [207]] Farha & Gall 2019 Li al.) 2020 2021b). Recent TCN variants
have incorporated advanced strategies including receptive field optimization (Gao et al. [021] Li
2024b 2025 Gao et al. 2022), boundary-aware mechanisms (Ishikawa et al. 2021 Wang
et al. 2020). multi-scale fusion (Singhania et al.) 2023), and diffusion models (Liu al. 2023a
Meanwhile, Transformer-bast
architectures have emerged as powerful alternatives, leveraging at- tention mechanisms for adaptive context modeling (Bahrami et al. 2023 Behrmann et al. 2022 Du
ct al. 2023) Wang et al. 2024 Yi al., 2021). However, limitations in privacy and robustness to ap- pearance variance hinder video-based methods, necessitating alternative modalities such skeleton sequence.
Skeleton-based Temporal Action Segmentation. STAS methods leverage skeleton data capture motion dynamics while remaining decoupled from visual appearance. Traditional approaches typi- cally adopt a two-stage pipeline combining Graph Convolutional Networks (GCNs) for spatial mod- cling and Temporal Convolutional Networks (TCNs) for long-range temporal dependencies (Filtjens) al. 2022, Ghosh et al., 2020). Recently, significant advancements have focused refining this architecture to handle complex motion patterns. DeST (i et al. 2023d) decouples mentio- modeling mitigate feature wer-shnoh,ing, while LAC (Yang a 2023) explores latent motion primitives. Most recently, LaSA (Ji et al., (024) integrates language priors for enhanced represen- tation learning. However, these methods primarily address architectural or semantic aspects, often overlooking the intrinsic synergy between tasks. Unlike current decoupled paradigms, explicitly construct a geometric bridge to restore cross-task collaboration within the feature space.
Task Decoupling in STAS. The conflict between classification and regression tasks, initially ob- served in object detection, has led to a widespread adoption decoupled classification and localiza- tion heads in object detectors (Wu et al. 2020: Ge et al. 202I). Similar to object detection, existing STAS methods (Li et al.) 2023d] Ji et al. (2024) employ separate heads for action classification and boundary localization with shared input feature. However, task decoupling paradigm suffers from

---

Published as a conference paper at ICLR 2026
two critical limitations: the separate decoder heads fail enable explicit cross-task collaboration, and the shared encoder with identical imput features leads to suboptimal trade-offs between compet- ing task requirements. To address these limitations, maintains task-specific feature adaptation and enables effective synergies through curvature-based geometric collaboration and dual-expert feature enhancement.
propose framework that simultaneously
METHOD
3.1
PROBLEM SETTING
Given a skeleton sequence F. E Pi, obtained from motion capture systems or pose esti- mation algorithms, where Din, T, and V represent the number of channels(where Di is 3 for 3D coordinates), frames, and joints respectively. The goal of skeleton-based temporal action segmenta- tion is predict the action label for each frame, with ground truth labels Y E {1, C}² where C' denotes the number of action classes.
3.2 FOUNDATION FRAMEWORK
The STAS task can be decomposed into two sub-tasks: action classification and boundary localiza- tion. We follow the typical backbone architecture from DeST (Li et al. 2023d) and LaSA (Ji et al.) 2024]. which employs spatial-tempora. modeling with dual decoders for classification and localiza- tion.
Spatial Modeling. Given an input skeleton sequence F., spatial feature gee extraction is per- formed by multi-scale graph convolution network (MS-GCN), which captures joint correlations at different receptive fields. For each hop kt an adjacency matrix A E 10.1)* VVV is constructed based the shortest joint distance, and normalized multi-scale adjacency AMs 10, 1) is formed by concatenating all K hops. The multi-scale spatial features are computed as:
F ggen ReLUI(AMS + FB),W.)).
(1)
where B VV(-VV a trainable matrix for adaptive joint relationships and W E RIXIx(X-D)xD handles channel adjustment. Based on DeST, we integrate the joint-wise skele- tal features F gen into the global frame-wise feature FsT through spatio-temporal attention fusion with joint embeddings. This module yields spatially enhanced frame-wise joint representations.
Temporal Modeling. For temporal modeling, in DeST and LaSA, Linear Transformer is adopted to achieve global temporal dependencies with Onn) complexity. The /-th linear transformer layer computed as:
F4+1 (GLLU)(())( )
W, + F'rr),
(2)
where Qu, K,, V, are query, key, value matrices derived from frame feature FST E RxT and 66-) denotes the sigmoid activation function.
Decoder Head. The final spatio-temporal features FST E DxT from the last layer of encoder fed into two distinct heads to obtain class prediction logits Y e RCXT and boundary pre- diction logits Y RIXT_ The predictions are further refined by segmentation branch (Linear Transformer) and boundary branch (TCN).
Frame-level cross-entropy loss and segment-level smoothness loss are applied to the class predic- tions:
T
1 Σ (e!
L- -
[log
(0rL DE.))
(3)
TC t=
cm
where jef denotes the predicted probability of ground truth label at time f. For boundary localiza- tion, binary logistic regression loss is utilized:
T
Lb
Σ
[v log (ii) + (1 v() log (1 u!)]
(4)
where of is the boundary ground truth (1 for boundary frames, otherwise). where
4

---

Published as a conference paper at ICLR 2026
3.3 CURVATURE-GUIE SYNERGY
Motivation. Existing STAS methods suffer from limited cross-task collaboration, treating classifi- cation and localization independent processes despite their inherent correlations. To address this limitation, we leverage a fundamental geometric principle from representation learning (Parulekar al,, 2023] Wang et al.) 2022b) when classification representations successfully separate differ- ent action classes, they confine action sequence trajectories within compact, class-specific regions. As formally derived in Appendix [ this geometric constraint implies that the average curvature of random walk is inversely proportional to the radius of its bounding hyper-sphere. This spatial confinement creates a fundamental geometric phenomenon: intra-segment points must frequently change direction to remain within their class-specific boundary, resulting in high curvature, while inter-segment points exhibit low curvature as they move between class regions. This geometric prin- ciple establishes natural bridge between classification quality and boundary detection, creating synergistic relationship that enhances both tasks simultaneously.
Curvature Calculation. We define the representational trajectories using the frame-wise classi- fication features Fels E
12 DxT, Specifically, F Feie is obtained by decoupling the global encoder output FST through the Expert Decoupling (EDD) module (refer to Sec, 3.44 for details). Note that since spatial pooling is performed within the encoder, Fats represents global frame-level features, implying that curvature computed globally per frame rather than per joint.
For each timestamp f, we select three consecutive points F Fele,t-r, Fic. and F Fels. along the tra- jectory, where the window size w is set ensure stability and adaptability across different temporal scales. The turning angle O, between two consecutive difference vectors is computed as:
8: = arccos (Fel, F Fele,t-)) (Fels,t4w - Fels,))
(5)
|F Fels,t-wl Fels,tt - Fat.,el'
where 0r E [0, x quantifies the instantaneous direction change at timestamp t. The curvature Ky is defined as the turning angle normalized by the sum difference vectors:
Kr = 8/(([ cls.t Fels,t-wll || Fels,tt
Feis.ell +||l+++0 c),
(6)
where e avoid division by zero. Subsequently, we compute the curvature- -ased boundary change metric from the obtained curvature sequence. We apply moving average smoothing to the raw curvature sequence reduce noise and enhance temporal consistency, noted R. The smoothed
curvature sequence is then normalized to ensure scale invariance A; marr(()) (oeu(( [Re]
and we obtain the boundary change metric by inverting the normalized curvature values, C. = R..
Re Re-min(Ru)
Curvature- Based Task Collaboration. To enable explicit collaboration between classification and localization tasks, we impose a bidirectional consistency constraint between the boundary prediction probabilities:
T MSE()).P(C.) + MSE(C, ((.)).
Leure T
where (SE(.)) denotes mean squared error distance and (p.) denotes the gradient stop function. This bidirectional constraint creates synergistic training strategy where the localization branch aligns with geometric properties from classification features, while classification learning benefits from boundary-aware supervision, resulting in mutually enhanced performance.
3.4 EXPERT-DRIVEN DECOUPLING
While our cervature-guided synergy mechanism establishes effective cross-task collaboration, its effectiveness depends critically on the quality of features used for curvature computation. However, both decoder heads currently receive identical shared encoder outputs, which are inherently com- promised representations that may not fully serve the distinct requirements of classification and lo- calization tasks. To this end, inspired by recent multi-modal perception tasks (Kim et al. 2025). our approach introduces an expert-driven decoupling mechanism, where the experts process the same encoder features but focus on different task- specific aspects. Specifically. we build two specialized maatio-temprra expert modules: classification experts and localization experts, which adaptively weight and refine encoder features according to their respective task requirements. Then, we use experts specializing in classification tasks example illustrate this. For spatio modeling.
5

---

Published as a conference paper at ICLR 2026
given the skeleton feature sequence FST D in the spatio-temporal encoder, we capture task-relevant spatial features to recalibrate each joint by computing joint attention vector via an SE-style block(Hu et al. 2018):
F Fs Esr + (LL
DDXXXV
where Zet is obtained by globally pooling FST over its temporal dimension. Then, we apply decou- pled spatio-temporal interaction (DSTI) layen[Li al., 2023d; Jī al. 2024), yielding the feature map FST E R RDXT" For temporal modeling, we deploy series of Gaussian experts extract task- relevant information across the global temporal distribution. Given that untrimmed videos contain multiple action segments, our approach systematically processes temporal information through the following steps: For a T'fframe video, we first uniformly divide into M segments, where M >> N (the number of action segments in the video). Each segment contains S [T/M] frames, and the m-th segment feature F(") E PxS can be divided from the tempiaal feature FST E IR DxT. This segment-basee approach allows the Gaussian experts to learn relative temporal patterns (e.g.- "the start of event") within normalized local context, rather than absolute positions in long video, which significantly simplifies learning and enhances generalization.
For each segment m, generate G Gaussian functions to assign adaptive weights the S frames within that segment. Each Gaussian expert is parameterized by learnable center 14 (m)) and variance (mm)
of (m) glm) Nnu, (o; (o."") )²). For simplicity, the normalized center 14 (m) and variance (m)
computed by a multilayer perceptron with nonlinear activation layers.
Instead of discrete expert selection, adaptively weight their contributions based on task-specific requirements. Then the router assigns routing values for each expert based on segment features ((m);
model uses Gaussian experts soft temporal masks that
(m) (eLPood(MML(Agg(PPP)
W E RC.
where Avg(-) denotes average pooling at temporal dimension, and W E DxG is learnable weight matrix that dynamically controls the influence of each expert. Subsequently, outputs from all experts are combined through weighted summation, the integration expressed as:
((mm
(m) (m) p(m)
(10)
The resulting expert weights enable the model to more effectively emphasize task-relevant informa- tion, capturing task-specific insights.
3.5 OVERALL OPTIMIZATION OBJECTIVE
Based on the proposed scheme, the overall optimization objective of the STAS model can be de- signed as follows:
L= Lb + XLeures
(11)
where À is parameearreettr for balance loss functions.
EXPERIMENT
4.1 DATASET AND EVALUATION METRICS
Datasets. Following DeST Li et al. 2023d) and LaSA (Ji et al. 2024). we evaluate on four stan- dard temporal action segmentation datasets: MCFF-22/MCFSS
(Liu et al.) 2021) (22/130 figure skating actions), PKU-MMD (Liu et al., 2017) (52 daily activities with X-sub/X-view splits), and LARa (Niemann et al. 2020) (8 warehouse activities). We follow standard protocols with 5-fold cross-validation for MCFS and single-split evaluation for PKU-MMD and LARa.
Evaluation Metrics. We report three standard metrics: (1) Frame- wise Accuracy (Acc), (2) Seg- mental Edit Score (Edit), and (3) Segmental FI Score (FI @ 10, 25, 50) at IoU thresholds of 10%, 25% and 50%. Segmental metrics provide comprehensive evaluation by penalizing over- segmentation errors.

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of methods on PKU-MMD and LARa datasets
PKU-MMD (X-sub)
PKU-MMD (X-view)
LARa
Model
Ace
Edit @1@ (10, 25, 50)
Ace
Edit F1@ [10, 25, 50
Acc
Edit
F1
(10, 25, 50)
MS+TCN++ Li al. 2023b
66.0 66.7
69.6
65.1
51.5
58.4
56.7
58.7
53.2
38.7
71.7
58.6
60.1
58.6
47.0
ETSN Li 2021b
68.4
67.1
70.4
65.5
52.0
60.7
57.6
62.4
57.9
44.3
71.9
58.4
64.3
60.7
48.1
ASRF Ishikaa
2021
67.1
72.1
68.3
56.8
60.4
59.3
62.5
58.0
46.1
71.
63.0
68.3
65.3
53.2
MS-GCN [Fittchs or
2022
70.0 65.8
68.5
63.9
50.1
66.5
64.0
67.1
62.4
49.9
73.7
58.6
63.8
59,4
47.6
MTST-GCN Clar
2024
20.0
65.8
68.5
63.9
50.1
66.5
64.0
67.1
62.4
49.9
73.7
58.6
63.8
59.4
47.6
ME-ST (Ji Ji al: 2025
74.1
70.5
76.6
73.2
62.4
68.5
67.2
72.3
68.8
58.1
74,2
65.0
71.0
68.2
57.1
DeST-TCK [Li al. 2023d
66.3
71.7
68.0
55.5
58.2
63.2
59.2
47.6
72.6
63.7
69.7
66.7
55.8
DeST al. 2023d
70.3
69.3
74.5
71.0
58.7
67.3
64.7
69.3
65.6
52.0
75.1
64.2
70.3
68.0
57.7
+Ours
71.8
69.9
75.4
71.8
59.2
67.9
65.4
69.9
66.3
52.4
76.1 65.6
71.7
68.8
58.7
LaSA (ji 2024
73.5
73.4
78.3
74.8
63.6
69.5
67.8
72.9
69.2
57.0
75.3
65.7
71.6
69.0
57.9
+Ours
74.3
74.4
79.3
76.3
65.5
71.0
68.9
74.4
70.7
58.0
76.6
66.2
72.5
70.0
59.0
Table 2: Comparison of methods on MCFS-22 and MCFS- 130 datasets
MCFS-22
MCFS-130
Model
Acc
Edit
Fl @
[10, 25, 50)
Acc
Edit
F1 @ {10, 25, 50)
MS-TCN (aarha & Gall 2019
75.6
74.2
74.3
69.7
59.5
65.7
54.5
56.4
52.2
42.5
ASTFormer (Y) al. 2021
78.7
82.3
82.8
77.9
66.9
67.5
69.1
68.3
64.0
55.1
MS-GCN (Filljens cl al 2022
75.5
72.6
75.7
70.5
57.9
64.9
52.6
52.4
48.8
39.0
ID-GCN+ASRE (Filtiens et al. 2022
78.1
81.6
86.4
83.4
73.0
67.1
68.2
68.7
65.6
56.9
DeST (Li et al.
2023d
80.4
85.2
87.4
84.5
75.0
71.4
75.8
75.8
72.2
63.0
LaSA J1 al.
2024
80.8
86.7
89.3
86.2
76.3
72.6
79.3
79.3
75.8
66.6
Ours
81.2
87.7
89.7
86.6
76.7
73.1
79.8
80.0
76.6
66.7
4.2
IMPLEMENTATION DETAILS
All experiments are conducted on single 3090 GPU with Adam optimizer. We set batch size to 1 and learning rate to 0.0005 for MCFS-22/12 training for 300 epochs. For PKU-MMD and LARa datasets, we use batch size and with learning rate 0.001, training for 300 and 40 epochs temporal window size of 10 for curvature computation. The loss balancing coefficient À is set to 4 for the PKU-MMD dataset, 2.5 for the LARa dataset, and network, each video is divided into 64
respectively. Besides,
for the MCFS datasets. In the temporal Mixture-of-Eie segments, and 2 Gaussian experts are generated for each segment.
4.3 COMPARISONS WITH THE STATR-OF-THE-ART.
We benchmark our method against state-of-the-art approaches, surpassing current methods' perfor- mance across all evaluated datasets (Tab. [ and Tab. 2 Notably, the most substantial gains observed in segmental Fl scores, directly validating that our method enhances temporal boundary precision. Crucially, this is not one-way benefit. The concurrent improvement in frame-wise curacy demonstrates the reciprocal nature of our design: refining boundaries leads to purer feature representations, which in turn benefits the classification task, confirming truly synergistic loop.
Table Ablation Study Results on PKU and LARa dataset
PKU-MMD (X-sub)
PKU-MMD (X-view)
LARa
Model
Acc
Edit
F1
(10, 25, 50}
Ace
Edit
F1@
[10, 25, 50)
Acc
Edit
FI@
[10, 25, 50)
base
73.5
73.4
78.3
74.8
69.5
67.8
72.9
69.2
57.0
75.3
65.7
71.6
69.0
57.9
CGS
74.3
78.7
76.0
70.9
69.1
74,5
76.2
65.9
72.1
58.7
73.8
78.3
69.9
68.2
74.2
57.4
76.2
65.8
72.0
58.4
Ours
74.3
74.4
79.3
76.3
65.5
71.0
68.9
74.4
58.0
76.6
66.2
72.5
70.0
59.0

---

Published as a conference paper at ICLR 2026
70
Score 73
Score
1180
1180
Figure 3: Haper-parameters analysis. 'w": window size; 'X': the CGS loss weigt; 'M" :he granular-
ity of EDD mmalysis;'** the number of Gaussian experts.
4.4 ABLATION STUDIES
We conduct ablation studies the LaSA Ji et al. 20224) baseline to dissect the contribution of our two core components: Expert-Driven Decoupling (EDD) and Curvature- Synergy (CGS), As shown in Tab. each component is both independently effective and mutually reinforcing.
Effectiveness of Expert-Driven Decou- pling (EDD). Adding only the EDD module (base+EDD) yields a consistent performance boost across all metrics. This confirms hypothesis that providing specialized features via task experts creates superior foundation for both classification and localization, over- coming the inherent compromises of a shared feature representation.
Table 4: CGS ablation studies on LARa dataset.
Metric
Acc
Edit Fl @ { {10, 25, 503
Base
75.3 65.7 71.6 69.0
57.9
Euclid
76.0 65.1 71.7 68.9
57.8
Cosine
75.2 65.2
71.5 68.6
57.0
Grad. Saliency
74.4
64.3
70.7 67.9
57.1
Effectiveness of Curvature-Guided Synergy (CGS). Independently adding the CGS module
Cury
76.2
65.9
72.1 69.4
58.7
(base+CGS) brings the most substantial gains the segmental F1 scores. This directly validates that our curvature-gus -guuddd loop is highly effective at enhancing boundary precision. The concurrent rise in accuracy further demonstrates the mechanism's 's reciprocal benefit: better boundary priors lead to better-clustered features for classification.
Synergy of the Full Model. Finally, our full model, combining both components, achieves the best results and demonstrates clear synergistic effect-it performance gain significantly surpasses the simple sum of the individual modules' contributions. This validates our core design philosophy: EDD provides the high-quality, specialized features that as optimal foundation, allowing the CGS module to then leverage these refined inputs to its full geometric potential for precise segmentation. We note that on PKU-MMD (X-view), CGS-only marginally leads in Edit and loose @ 10 scores. This implies that under drastic view shifts, the smoother shared representations of CGS favor prediction continuity. However, the full model dominates on strict metrics (FI @ (25, 50)) and Accuracy, verifying that EDD's specialized features are indispensable for sharpening boundaries and distinguishing fine- me--graieed semantics.
4.5
COMPONENT ANALYSIS
4.5.1
CURVATURE-GUINE SYNERGY ANALYSIS.
Analysis of Synergy. Our ablation study (Tab. 6 dissects the CGS module within the overall framework. Isolating the forward path (C-L) primarily boosts Fl scores by refining boundaries, while the backward path (I.+C) enhances accuracy by regularizing features. Crucially, the full CGS module yields synergistic gain significantly larger than the sum of its parts, confirming that boundary guidance and feature refinement mutually reinforcing.
Superiority of Curvature-based Guidance. We validate our approach against traditional distance metrics (Euclidean, Cosine) and learned boundary detectors eeg. gradient-based saliency) in Ta- ble Results demonstrate that curvature consistently provides the best guidance. Existing baselines show limitations: distance metrics are sensitive feature magnitude, while saliency maps tend to highlight discriminative action centers (c.g., peak motion) rather than temporal boundaries. con- trast, curvature explicitly captures the directional evolution of the feature manifold. This geometric
8

---

Published as a conference paper at ICLR 2026
prior serves precise, parameter-free proxy for state transitions, remaining robust to both subtle drifts and abrupt changes.
Curvature Direct Boundary Proxy. To further validate our premise, we use the in- verted curvature values directly as boundary predictions on Lara dataset. This simple threshold-based method achieved remarkably as shown in Tab. [2] This strongly confirms that low-curvature points in the feature trajec- tory are highly effective proxy for temporal boundaries, validating the foundational princi-
Table 5: Refine with different boundary predic- tion.
Boundary
Ace Edit F1@ {10, 25, 50}
Pred
76.6 66.2 72.5 70.0
59.0
Curv
75.4 64.6 72.3 69.4 55.1
ple of our CGS module.
Table Task synergy ablation study on LARa
Table 7: EDD ablation study on LARa dataset.
Metric
Acc
Edit F1 @ [10, 25, 50}
Metric
Acc
Edit
Fl @ { [10, 25, 50}
Base
75.3 65.7 71.6 69.0
57.9
Base
75.3 65.7 71.6 69.0 57.9
C-L
76.1 65.3 71.7 68.8 58.2
Indep. Enc.
75.7 64.5
71.6 68.3
57.7
L-C
75.9
65.4
71.6 69.1
58.1
Pyr. Decoupling
75.6 64.7
71.2 68.4
57.9
CGS
76.6
66.2
72.5 70.0 59.0
EDD
76.2 65.8
72.0
69.4
58.4
4.5.2 EXPERT-DRIVEN DECOUPLING ANALYSTS
Advantage of Dynamic Decoupling, We benchmarked our EDD against alternative decoupling strategies in Tab. 7 on Lara dataset. Our approach is superior because avoids two common pitfalls: it neither compromises features through forced sharing (Base model) nor loses valuable signals through complete isolation (Indep. Enc.). Unlike static allocation (Pyr. Decoupling), our expert routing mechanism dynamically creates specialized features tailored to each task's 's needs. This allows the model to achieve the best of both worlds: generating highly discriminative, task- specific representations while still enabling beneficial, selective information exchange.
4.6 HYPERPARAMETE ANALYSIS
We conduct brief analysis key hyper-parameters in Fig. 3 including curvature window size the CGS loss weight X, the granularity of EDD analysis M, and the number of Gaussian experts
The hyperparameter À balances the contribution of our Curvature Guidance Synergy (CGS) loss. Empirically, the performance peaks at λ=2. smaller value provides insufficient guidance, while a larger value overly constrains the model and hurts performance. Segment Count M controls the tem- poral resolution. The model achieves the best results with M = 64, striking an optimal balance. A smaller M offers poor temporal precision, whereas larger M causes context loss within each short segment. Gaussian Expert Count G sets the modeling capacity within each segment. Performance is optimal 2, while more experts (G 2) lead to diminishing returns and potential overfitting. Hyper-paramette w controls the temporal receptive field. We found w=10 optimal, balancing noise suppression and boundary sensitivity. Smaller windows (w=5) capture insufficient context, while larger ones (w >= 40) over-smooth the sharp directional changes that define action boundaries. Detailed hyperparameter analysis provided in Appendix
4.7 ERROR ANALYSIS AND LIMITATIONS
Our analysis identifies two scenarios where geometric priors face intrinsic challenges. First, low- dynamic actions (e.g., Step Sequences) exhibit continuous feature evolution without abrupt shifts. This results shallower curvature valleys compared to high-dynamic actions (e.g., Jumps), making boundary localization physically more ambiguous Additionally, skeleton sequences inevitably con- tain sensor noise or estimation jitter. This can introduce high-frequency fluctuations in the feature trajectory, creating spurious curvature peaks (false positives) in non-boune non-boundary regions, shown in Fig. 4 However, despite these geometric constraints, quantitative results (Tab. 8 demonstrate that method remains highly robust. Notably, achieve larger relative improvement over the

---

Published as a conference paper at ICLR 2026
Category
Metric
Base
Ours
@
71.18
72.86
Lavback Soin internal transition.
High-Dynamic
Edit
58.44
61.61
GT
@ 10
49.62
52.86
Oux
Low-Dynamic
Edit
21.81
27.83
(b) Transition: NONE - Step Seg.
Table 8: Performance on High vs. Figure 4: Visualization of Limitation Cases. (a) Internal Low Dynamic Actions. Low-dynamic spin variations cause noise. (b) Gradual step transitions actions show lower absolute scores but lack deep curvature valleys.
higher relative gains from our method.
baseline on these challenging low-dynamic actions (+6.02% Edit Score) compared to high-dynamic ones (+3.17%). This indicates that even weak curvature signals act as effective differential cucs, providing structural constraints that guide the model to resolve ambiguities where pure semantic baselines typically fail.
4.8
QUALITATIVE COMPARISON
Fig. 5 shows qualitative comparisons with previous methods, the top row shows the curvature vi- sualization results of the skeleton sequence features. Our approach utilizes curvature- based task collaboration and Expert-driven adaptive feature extraction to achieve more accurate action predic- tion and boundary detection. The curvature mechanism enables effective cross-task synergy through geometric feature properties, while specialized experts provide task-specific representations for clas- sification and localization. These improvements result in segmentation outputs that closely match ground truth annotations.
Gan
Oes
Oers
1a5n
B:57
(a) PKU-MMD (X-sub)
(b) LARa
(c) MCFS-130
Figure 5: Qualitative comparison of segmentation results PKU-MMD (X-sub), LARa, MCFS- 130. Our method alleviates over-segmentation and achieves more precise action boundary local- ization, demonstrating better temporal consistency and robustness compared to baseline methods.
CONCLUSION
In this work, we propose novel framework for skeleton-hased temporal action segmentation that addresses the fundamental limitation of insufficient cross-task collaboration. Our approach intro- duces survature-based task synergy mechanism that exploits geometric properties of feature quences to establish effective collaboration between classification and localization sub-tasks, cou- pled with a dual-expert weighting mechanism within Mixture of Experts framework for task- adaptive feature extraction. However, our method still faces limitations in handling extremely noisy skeleton data and complex multi-person scenarios, leaving room for further improvement. Future work should explore more robust geometric priors and investigate the integration of multi-modal information to enhance the framework's applicability in real-world deployment scenarios. See Ap- pendix D for reproducibility statement and Appendix E for our statement on LLM usage.
ACKNOWLEDGMENTS
The authors gratefully acknowledge the financial support from the National Natural Science Foun- dation of China (NSFC). This work was supported by NSFC under Grant Nos. 62437001 and 62506040.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Emad Bahrami, Gianpiero Francesca, and Juergen Gall. How much temporal long-term context is needed for action segmentation? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10351-10361, 2023.
Nadine Behrmann, S Alireza Golestanch, Zico Kolter, Juergen Gall, and Mehdi Noroozi. Unified fully and timestamp supervised temporal action segmentation via sequence to sequence transla- tion. In European conference on computer vision. pp. 52-68. Springer, 2022.
Shurong Chai, Rahul Kumar Jain, Jiaqing Liu, Shiyu Teng, Tomoko Tateyama, Yinhao Li, and Ven-Wei Chen. A motion-aware and temporal-enhanced spatial-temports graph convolutional network for skeleton-based human action segmentation. Neurocomputinng 580:127482, 2024.
Diao, C Ernst A Montemayor, and Ziegler. Curvature of random walks and random polygons in confinement. Journal of Physicy A: Mathematical and Theoretical, 46(28) jun 2013. doi: 11119///775-1111///8/88/88 46/28/285201
Li Ding and Chenliang Xu. Tricorett: A hybrid temporal convolutional and recurrent network for video action segmentation. arXiv preprint http:ppp///oa.ooooooo/0000011/11
2017.
Dazhao Du, Enhan Li, Lingyu Si, Fanjiang Xu, and Fuchun Sun. semestamp-superriseee action segmentation in the perspective clustering. arXiv preprint arXiv:2212. 1694, 2022.
Dazhao Du, Bing Su, Yu Li, Zhongang Qi, Lingyu Si, and Ying Shan. Do we really need temporal convolutions in action segmentation? In 2023 IEEE International Conference on Multimedia and Expo (ICME), pp. 1014-1019. IEEE, 2023.
Yazan Abu Farha and Jurgen Gall. Ms-ten: Multi-stage temporal convolutional network for ac- tion segmentation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 3575-3584, 2019.
Benjamin Filtjens, Bart Vanrumste, and Peter Slacts. Skeleton-based action segmentation with multi-stage spatial-temporal graph convolutional neural networks. IEEE Transactions on Emerg- ing Topics in Computing, 12(1):202-22 212, 2022.
Shang-Hua Gao, Qi Han, Zhong-Y Yu Li, Pai Peng. Liang Wang, and Ming-Ming Cheng- Global2local: Efficient structure search for video action segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16805-16814, 2021.
Shanghua Gao, Zhong-Yu Yu Li, Qi Han, Ming-Ming Cheng, and Liang Wang. Rf-next: Efficient
receptive field search for convolutional neural networks. IEEE transactions on pattern analysis and machine intelligence, 45(3):2984-300 3002, 2022.
Zheng Ge, Songtao Liu, Feng Wang. Zeming Li, and Jian Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08.. 08430 2021.
Pallabi Ghosh, Yi Yao, Larry Davis, and Ajay Divakaran. Stacked spatio-temporal graph convolu- tional networks for action segmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 576-585, 2020.
Jie Hu, Shen, and Gang Sun. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and paltern recognition, pp. 7132-7141, 2018.
Yuchi Ishikawa, Seito Kasai, Yoshimitsu Aoki, and Hirokatsu Kataoka. Alleviating over- segmentation emrors by detecting action boundaries. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 233212331, 2021.
Haoyu Ji, Bowen Chen, Xinglong Xu, Weihong Ren, Zhiyong Wang, and Honghai Liu. Language- assisted skeleton action understanding for skeleton- based temporal action segmentation. In Euro- pean Conference Computer Vision, pp. 400-417, Springer, 2024.

---

Published as a conference paper at ICLR 2026
Haoyu Ji, Bowen Chen, Wenze Huang, Weihong Ren, Zhiyong Wang, and Honghai Liu. Snippet- aware transformer with multiple action elements for skeleton-based action segmentation. IEEE Transactions on Neural Networks and Learning Systems, 36(9):77422-1747, 2025. doi: 10. 2020TTNNN.S.2255556630000
Hongyeob Kim, Inyoung Jung, Dayoon Suh, Youjia Zhang, Sangmin Lee, and Sungeun Hong. Question-aware gaussian experts for audio-visual question answering. In Proceedings of Com- puter Vision and Pattern Recognition Conference, pp.
2025.
Colin Lea, Michael D Flynn, Rene Vidal, Austin Reiter, and Gregory D Hager. Temporal convolu- tional networks for action segmentation and detection. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 156-165, 2017.
Jac-Gil Lee, Jiawei Han, and Kyu-Youn Whang. Trajectory clustering: a partition-and-grome framework. SIGMOD '07, pp. 593-604, New York, NY, USA, 2007. Association for Computing Machinery. ISBN 9781595936868. doi: 10. (0.114//247480.1247555 1247546.
Guozhang Li, Jie Li, Nannan Wang. Xinpeng Ding, Zhifeng Li, and Xinbo Gao. Multi-hierarchical category supervision for weakly-supervised temporal action localization. IEEE Transactions Image Processing, 30:9332-9344, aa doi: 10. 10.110991122222..12267..
Guozhang Li, De Cheng, Xinpeng Ding, Nannan Wang, Xiaoyu Wang, and Xinbo Gao. Boost- ing supervis temporal action localization with text information. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10648-10657, June 2023a.
Guozhang Li, De Cheng, Xinpeng Ding, Nannan Wang. Jie Li, and Xinbo Gao. Weakly su- pervised temporal action localization with bidirectional semantic consistency constraint. IEEE Transactions on Neural Networks and Learning Systems, 3509)113032-3344,, 2024a. doi: 10. 10.11099TNNLS.2223.00 222.66666.....
Guozhang Li, De Cheng, Nannan Wang, Jie Li, and Xinbo Gao. Neighbor-guided pseudo-label generation and refinement for single-frame supervised temporal action localization. IEEE Trans- actions on Image Processing, 33:2419-2430, 2024b. doi: 10. 10.1109//PP2222433388777
Guozhang Li, Xinpeng Ding, De Cheng, Jie Li, Nannan Wang, and Xinbo Gao. Etc: Temporal boundary expand then clarify for weakly supervised video grounding with multimodal large lan- guage model. IEEE Transactions on Multimedia, 1111172277782 2025. doi: 10. 10.1009/MM..024. 3521758.
Shi-Jic Li, Yazan AbuFarha, Yun Liu, Ming- Ming Cheng, and Juergen Gall. Ms-ten++: Multi-stage
temporal convolutional network for action segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2020. doi: 10.1109/TPAMM 109//TPAM1122000300155555 2020.302156.
Shijie Li, Yazan Abu Farha, Yun Liu, Ming-Ming Cheng, and Juergen Gall. Ms-ten++: Multi-stage temporal convolutional network for action segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):6647 6658, 2023b. doi: 10.1 0009/TPPM..220030211111
Yun-Heng Li, Yuin Liu, Shn Liu, Lin Feng, and Hong Qiao. Involving distinguished temporal graph convolutional networks for skeleton-based temporal action segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 34(1):647-660, 2023c.
Yunheng Li, Zhuben Dong, Kaiyuan Liu, Lin Feng, Lianyu Hu, Jie Zhu, L. Xu, Shenglan Liu, et al. Efficient two-step networks for temporal action segmentation. Neurocomputing, 454:373-381, 2021b.
Yunheng Li, Zhongyu Li, Shanghua Gao, Qilong Wang, Qibin Hou, and Ming-Ming Cheng. A decoupled spatio-temporal framework for skeleton-basee action segmentation. arXiv preprint arXiv:2312.058300 2023d.
Chunhui Liu, Yueyu Hu, Yanghao Li, Sijie Song, and Jiaying Liu. Pku-mmd: A large scale bench- mark for skeleton based human action understanding. In Proceedings of the workshop on visual analysis in smart and connected communities, pp. 1-8, 2017.
12

---

Published as a conference paper at ICLR 2026
Daochang Liu, Qiyue Li, Anh-Dung Dinh, Tingting Jiang, Mubarak Shah, and Chang Xu. Diffusion action segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 111199-0119,,. 2023a.
Kaiyuan Liu, Yunheng Li, Shenglan Liu, Chenwei Tan, and Zihang Shao. Reducing the label bias for timestamp supervised temporal action segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5503-6513, June 2023b.
Shenglan Liu, Aibin Zhang, Yunheng Li, Jian Zhou, Li Xu, Zhuben Dong, and Renhao Zhang. Temporal segmentation of gaie-g semantic action: A motion-centered figure skating dataset. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 2163-216222221 2171, 2021.
Friedrich Niemann, Christopher Reining, Fernando Moya Rueda, Nilah Ravi Nair, Janine Anika Steffens, Gernot A Fink, and Michael Ten Hompel. Lara: Creating a dataset for human activity recognition in logistics using semantic attributes. Sensors, 20(15):408 2020.
Advait Parulekar, Liam Collins, Karthikeyan Shanmugam, Aryan Mokhtari, and Sanjay Shakkot- tai. Infonce loss provably learns cuuster-preservngg representations. In Gergely Neu and Lorenzo Rosasco (eds.), Proceedings of Thirty Sixth Conference on Learning Theory, volume 195 of Pro- ceedings of Machine Learning Research, pp. 1914-1961. PMLR, 12-15 Jul 2023.
Ziyou Ren, Guozhang Li, Nan Cheng, Anqi Wu, Nannan Wang. and Xinbo Gao. Cluster assumption- guided mumnsttamp-spprvissee temporal action segmentation. IEEE Transactions Multimedia, 27:4206-4216, 2025. doi: 10. 10990MMM0002002333333111
Yooju Shin, Susik Yoon, Sundong Kim, Hwanjun Song, Jac-Gil Lee, and Byung Suk Lec. Coherence-based label propagation over time series for accelerated active learning. The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Yooju Shin, Susik Yoon, Hwanjun Song, Dongmin Park, Byunghyun Kim, Jae-Gil Lee, and Byung Suk Lee. Context consistency regularization for label sparsity in time series. In Interna- tional Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 31579-31595. PMLR, 2023.
Yooju Shin, Jachyun Park, Susik Yoon, Hwanjun Song, Byung Suk Lee, and Jae-Gil Lee. Exploiting representation curvature for boundary detection in time series. Advances in Neural Information 2024.
Processing Systems,
Bharat Singh, Tim K Marks, Michael Jones, Oncel Tuzel, and Ming Shao. A multi-stream bi- directional recurrent neural network for fine-grained action detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1961-1970, 2016.
Dipika Singhania, Rahul Rahaman, and Angela Yao. C2f-ten: A framework for semi-and fully- supervised temporal action segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence,
2023.
Jiahui Wang, Zhengyou Wang, Shanna Zhuang, Yaqian Hao, and Hui Wang. Cross-enhancement transformer for action segmentation. Multimedia Tools and Applications, 83(9):25643-25656, 2024.
Yifei Wang, Qi Zhang, Yisen Wang. Jiansheng Yang, and Zhouchen Lin. Chaos is ladder: A new theoretical understanding of contrastive learning via augmentation overlap. ArXiv, 13457, 2022a. URL https::/app semant i cschol ar. orgoo//rorrrsss:
247748621
Yifei Wang, Qi Zhang, Yisen Wang, Jiansheng Yang, and Zhouchen Lin. Chaos is ladder: A new understanding of contrastive learning. In International Conference on Learning Representations, n forum? dd=ECggmVVyeUz
URL 2/2bb.uRnnnn:pp:://eeeeeeee
Zhenzhi Wang, Ziteng Gao, Limin Wang, Zhifeng Li, and Gangshan Wu. Boundary cascade networks for temporal action segmentation. In European Conference on Computer Vision, pp- 34-51. Springer, 2020.
13

---

Published as a conference paper at ICLR 2026
Yue Wu, Yinpeng Chen, Lu Yuan, Zicheng Liu, Lijuan Wang. Hongzhi Li, and Yun Fu. Rethinking classification and localization for object detection. In Proceedings of the IEEE/CVE conference computer vision and pattern recognition, pp. 10186-10195, 2020.
Sijie Yan, Yuanjun Xiong, and Dahua Lin. Spatial temporal graph convolutional networks for skeleton-based action recognition. In Proceedings of the AAAI conference on artificial intelli- gence, volume 32, 2018.
Di Yang, Yaohui Li, al. Latent action composition for skeleton- based action segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10000- 10010, 2023.
Fangqiu Yi, Hongyu Wen, and Tingting Jiang. Asformer: Transformer for action segmentation. arXiv preprint arXiv:2110.085600 2021.
Yang Zhao and Yan Song. Turning to teacher for timestamp supervised temporal action segmenta- tion. In 2022 IEEE International Conference on Multimedia and Expo (ICME), pp. 01-06, 2022.
doi: 10. 000/////CCCEE2222222222229999969
14

---

Published as a conference paper at ICLR 2026
Algorithm 1 Curvature-Guided Synergy (CGS)
Require: Mini-batch = {(X, Y, y" encoder/backbone & classification head Hels boundary
head Hs; window size w:; smoothing window m. weight X; small stop-gradient yp(-)
Ensure: CGS loss Leary and total loss Lrotal
1: for each sequence (X, Y. Y.) B do
2: F FST & {(X)
Spatial-temporal encoding (Sec. Foundation) Duul heads
3: Y - Hets (FsT). 8 (FsT
4: Fels t- frame-wise classification features from encoder/class head
5:
Curvature construction on classification trajectory
6:
for t = W...., T W do u, - Fel.t Fii Vt. f- Fels,t-w - Fole,t (Ur Vo)
7:
8:
0: <- arccos
Turning angle, cf. Eq. (7)
|ue [vell +
0₃
9:
Ke |1 |ive
Curvature, cf. Eq.
+
10: end for
MovingAveraggo(x; m)
Noise suppression
R4 min R)
12:
R max(R) min(k) (6) +
Min-max normalization
13:
C. + R2
Boundary-change metric from curvature
14:
Losses
15:
Le frame CE segment smoothness (Eq. (5)) Cb + binary logistic loss for boundaries (Eq. (6))
16:
17:
Leurv + ΣT [MS ((C)) + +(E(
Bidirectional consistency, cf.
Eq. (9)
18: end for
19: Listal &- Le + Ls + X Leurv
20: return Leure Lioual
A APPENDIX: ALGORITHM OF THE PROPOSED GURVSEG
A.1 CURVATURE-GUIDEE SYNERGY
Algorithm converts the Trame-wise classification features into a geometry-derived boundary prior and enforces bidirectional consistency with the boundary head. Specifically, we (I) build tempo- ral trajectory from classification features, (2) compute a k-step turning-angle curvature per frame, followed by smoothing and min-max normalization to obtain boundary-change metric, and (3) align this metric with the predicted boundary probabilities using a stop-gradient operator in both directions. The resulting curvature-based consistency loss is combined with the standard classifica- tion and boundary losses, yielding an self-supervised signal that couples classification quality and boundary localization while preserving model plasticity.
A.2 EXPERT-DRIVER DECOUPLING
Algorithm disentangles task-specific evidence for classification and localization while sharing common encoder. Two specialist expert stacks refine the encoder outputs in complementary man- ner: (i) a spatial recalibration module applies SE-style joint attention to emphasize task-relevant joints and then uses decoupled spatio-temporal interaction (DSTI) layer to form sequence fea- tures: (ii) a temporal mixture of Gaussian experts operates on uniformly partitioned segments of the sequence, where each expert produces soft temporal mask parameterized by learnable centers and variances. lightweight router predicts routing scores for all experts from segment descriptors, and the masked expert outputs aggregated by weighted sum. The resulting expert-refined features
15

---

Published as a conference paper at ICLR 2026
Algorithm 2 Expert-Driven Decoupling (EDD)
Require: Encoder spatial features F s,1. E RVVT, segment count M; experts G; small
Ensure: Expert-refined sequence F sr E IR DxT for target head (cls or loc)
I: Spatial recalibration (SE-style) and DSTI
2: 2, + GFP,(F...). ...))
D Global average over time
3: a < (M.P(Z..)
> is Sigmoid
4: Fs,r f- Fs. a Fst
Joint-wise attention b
5: FST f (F.)(F...)
F SST E R DxT
6: Temporal Gaussian experts on uniform segments
7: S < [T/M partition FsT into (F(m) E R P
mm=1
8: for m to M do
9:
f(m) Avg, A (F(m)) RRD
10: for = to G do
(n (m]m) o (m)
MLP, MLP,(f(m))
P Centers and spreads > Ensure positivity
12:
- Softplus(o))
(m) t
9 [m) [s] exp(
(w-p (m (n))y2
13:
for 1...S
2(x, 1
9. (m)
t gi (m) E5n
9 +
Normalize mask
H (m) Fim) Di
H(m)
(m)
Time-wise weighted features
16: end for
17: ((m) + of (MP(((m)))W' E RG
Routing scores
(m) !! ((m) + Eim Ti
18:
Soft aggregation across experts
19:
end for
20: F FST <- Concatm (f(m))
b Back to length-T sequence
21: return FST
emphasize relative temporal patterns within segments (e.g., onset, middle, end) rather than absolute timestamps, yielding stable gradients, better generalization, and clean decoupling for the two heads.
B APPENDIX: THEORETICAL SUPPLEMENT
B.1 REPRESENTATIONAL TRAJECTORIES
trajectory is the path an object in motion follows through space and time (Lee et al. 2007). In feature spaces, representation trajectory Fi (f.)<1> (f.) RD in feature space defined by sequential representations over time. Curvature at a point measures the instantancous rate directional change, quantifying how much curve bends from a straight line.
For three consecutive time points on trajectory in feature space, t t, ++ 1, their corresponding representations are respectively f,-1, f--1; fut1- Naturally, we can obtain two difference vectors,f, f andf,+1 - fg. he turning angle 6, between the two vectors is calculated similarly to Eq. 12
(fr f- 1) (fe fe) [.. f [[f f
0ft arccos
(12)
Each turning angle O₂ lies in [0. m. Thus, curvature is the rate at which the difference vector rotates per unit length-i.es the rate change of direction between two difference vectors-an and is given by Eq. 13
0
Kft (f.,,
(13)
f
[
fell +e)
16

---

Published as a conference paper at ICLR 2026
B.2 TRAJECTORY CURVATURE AND GEOMETRIC CONSTRAINTS
For an action subsequence belonging to a specific category C₁ its representation trajectory can be assumed to be constrained within hyper-sphere Sc; of radius R2 This assumption stems from a widely confirmed observation: Representation learning aims to construct a feature space with good class separability. The learning process naturally clusters samples of the same class together, resulting in distinct and compact class clusters. Well-embedded points within specific category (or segment) can be extracted from their category-specife sphere (Wang et al.) 2022a). In other words, the representation trajectory of intra-segment points can be constrained within category sphere, while the representation trajectory of inter-segment points cannot.
Consider transition from class C2 to another class Cj- Let be the other hyper-spheres of class C₁ with radius Ry. To simultaneously contain Sc., Sc, and the transition points between them, we define larger hyper-sphere with radius Rinter. By definition, it necessarily holds that R4 < Renter and Rg < Rinter-
2023), ,a simplifying Based on the temporal coherence inherent in time series (Shin al. 2022
assumption is introduced: A representation trajectory Fy is a Markov chain, the generation of the next point f, satisfies two conditions: (1) Equal-step property: f, sampled uniformly random from the unit hyper-sphere centered at f;-1; (2) Constrained- boundary property The entire trajectory contained within larger hyper-sphere of radius: ||6--||| < R.
The significance of this assumption is that the equal-step-size property (Condition 1) greatly sim- plifies the calculation of curvature. According to Eq. [3] when the step size (the denominator) constantly I, the curvature of the trajectory is equivalent to its turning angle O,. This allows to focus on analyzing the expected value the turning angle.
Based on the above model, we can define the mean curvature of a trajectory KF, (R) under specific constraints:
KFF (R)
1T
(14)
EET
where [ is the expectation of the curvature at timestamp t with respect to the distribution of the representations in the confining hyper-sphere radius R.
In more intuitive way, we analyze special case in 2-dimensional space. We define a simplified
model, f, is an arbitrary point on the border of confining circle S. In this case, Eq. 14 can be simplified as:
1 2
KF, (R)
arccos 2R
(15)
Specifically, Let the confining circle be centered at the origin O with radius R > 1, and let fo lie on its boundary, ||G|]| = R. Assume unit step size ||f, - f--ill ||f - fell 1, and choose f--1 on the line segment from O to fę so that the incoming direction aligns with the radial direction. The next point f++1 sampled uniformly on the unit circle centered at f,, restricted to lie inside the confining circle; equivalently, f lies on the intersection are of the two circles.
With unit step size, the discrete curvature equals the turning angle 8, E [0, .). Let C and consider triangle 4OCP with P = f at the circle intersection. Then ||OC|| = R, ||CP|| = 1, ||OP|| = R.
122 + 11111 R2
1
COS 1b
Vmmx = arccos
(16)
2R.
2R
2R
and by the cosine rule, for 1h 1OCP, Geometrically, the turning angle and s satisfy 0 = π th
hence the feasible range is 0
arccos
2R
Because fe+1 is uniform along the intersection arc, & is uniform on [0, mmax]. and thus 0 is uniform on [TT - Vmaxx r). Therefore, the average curvature (average turning angle)
Ke (R) = -EA =
arccos
(17)
2
2R
17

---

regularization against the primary ob- Published as a conference paper at ICLR 2026
Moreover,
d
RFr (R) dR
2R 2R (R> 1),
<
(18)
80 KFr (R) decreases monotonically with R.
Thus, when f lies the boundary of the smaller constraining circle, the mean curvature fę is higher. As R increases, the random walk has more space, so the trajectory less likely to hit the boundary. Consequently, when the trajectory remains inside the smaller circle, its mean curvature decreases. The above conclusion can be generalized to higher dimensions, and the proof of the conclusion is provided by Diao al. (2013).
A constraint circle's radius for an inter-segment trajectory is always larger than that for an intra- segment trajectory, because an inter- seeee-tegmett trajectory passes through two different quasi-spheres. Since Kiy (R) is decreasing function, KJy (Rintra) KJT (Rinter).
C APPENDIX: ADDITIONAL EXPERIMENTS
This section analyzes the impact of loss weight X, number of segments M, number of Gaussian experts G, and window size w on performance metrics based on the four tables in the graph (Tab. 12). Except for the hyperparameter being examined, all other settings are fixed at their default values.
Hyper-Parameter À balances the curvature/boundary-aawa
jective. Increasing from 2 consistently improves all metrics; further increasing to 2 yields slight degradation, indicating over- avortsin At À = 2 we obtain the best results: Acc 76.6, 72.5 5/70000/5... moderate curvature weight strengthens boundary
Edit 66.2, F1@ 10/25/ contrast while preserving intra- segment details; too large values over-regularize fine structures.
Plype-Paraametrr M controls the segment-level resolution (e.g.. codebook/partition capacity). Per- formance improves steadily from M 8 to M = 64 and slightly drops at M 128. 12e best configuration M 64 with the same top-line metrics as above. Small M underfits (insufficient capacity, mixing semantics); excessively large M increases variance (data sparsity per segment). M 64 strikes a favorable capacity-stabiln balance. Besides, an adaptive strategy that dynami- cally scales M based action density yields only marginal improvement the cost of significantly higher computational complexity.
Tlype-PParamete G sets the complexity of the Mixtureof--Gausee Gaussians expert module model multi- modal transitions. Moving from G = to G = 2 yields consistent improvements, while G 2 provides diminishing returns and slight drops. The best result is G 2. Two experts suffice to capture the dominant transition modes given the dataset size; larger G introduces redundancy and less stable estimation.
Hyper- Pareee-erretterr w defines the temporal context for curvature estimation/smoothingg or short-range attention. 10 is optimal: too small window (w lacks context, while large windows (w > 40) ovor-h and blunt boundary peaks. Boundaries typically manifest as short, sharp directional changes. moderate window suppresses intra-segment noise without washing out changepoints.
Table 10: Effect of segment count M on model Table 9: Effect of hyperparameter on model performance performance
M
Acc
Edit
F1 @ [10, 25, 50}
X
Ace
Edit
F1
@
10, 25,
50}
8
76.3
64.9
71.5 68.3
56.7
0
75.8
65.3
71.7
69.0
57.4
75.5
64.6
71.5 68.4
56.8
1
76.4
65.2
71.7
69.3
57.8
32
76.4 65.9
72.2 69.7
58.3
2
76.6
66.2
72.5
70.0
59.0
64
76.6
66.2
72.5
70.0
59.0
3
76.1
65.5
72.1
69.6
58.1
128
76.0
65.5
71.6
69.0
56.9
4
75.9
64.8
71.7
68.5
57.9
Adaptive
76.3 64.3
70.9
68.2
58.0
18

---

Published as a conference paper at ICLR 2026
Table 11: Effect of Gaussian expert count G on Table 12: Effect of window size w on model model performance
performance
G
Acc
Edit
F1
@
10, 25,
50}
W
Acc Edit FI@ {10, 25, 50)
76.3
65.4
72.0
69.7
58.4
5
76.3
65.1
71.1 67.9
56.7
2
76.6
66.2
72.5
70.0
59.0
10
76.6
66.2
72.5
70.0
59.0
76.4
65.0
71.7
68.7
58.1
20
76.0
65.3
71.8
68.6
57.0
75.8
64.4
71.4
68.3
57.4
40
76.2
64.5
71.4
68.9
56.8
75.0
65.3
70.9
67.8
57.6
80
76.2
64.3
70.3
67.4
56.3
[W /w
ry Vy
64
Epoch
Acc
Edit
F1@10 10
21
F1@50
128
7
/M W
2
74.0
73.2
78.6
75.6
64.0
GT
8
74.4
73.0
78.5
74.9
63.7
74.3
74.4
79.3
76.3
65.5
Figure 6: Curvature Improves Throughout Training. Early epochs (e.g.. 2, 8) exhibit noisy or flat curvature signals, while the converged model (Epoch 64, 128) exhibits distinct low- curvature valleys at action boundaries and high- curvature fluctuations within segments, validat- ing geometric assumption.
74.3
74.4
79.3
76.3
Table 13:
Training progression on PKU-
MMD (X-Sub). Curvature-guides features gain discriminability steadily as training proceeds.
Besides, feature quality is critical for curvature estimation. Untrained classification networks pro- duce noisy curvature, so we use staged training: the CGS module activates only after seveal warm-up epochs. We conducted a sensitivity analysis on to find the optimal balance between feature stabiliza- tion and joint optimization benefits. Our analysis reveals an optimal warm up epoch for curvature guidance. Starting CGS too early on underdeveloped features is ineffective, while proper con- figuration ensures the formation of robust feature clusters before curvature constraints are applied, effectively managing this dependency. As shown in Fig. 6 and Tab. [3] ( (Appendix C), early curva- ture noisy but begins reflecting action boundaries. With continued training, feature compactness improves, further enhancing curvature quality. The gains saturate after certain number of warm-up rounds.
D APPENDIX: REPRODUCIBILITY STATEMENT
We detail the model and training setup Section and present the dataset, pre-processing, and evaluation protocol in Section 4 All syper-parammters and computational details are reported in Appendix C. The code included the supplementary material.
E APPENDIX: LLM USAGE STATEMENT
We utilized a large language model (LLM) to improve the grammar, clarity, and overall readability of this manuscript. The LLM's role strictly limited to language editing and polishing. All scientific contributions, including the core ideas, methodology, experimental design, data analysis, and conclusions, the original work of the human authors. The use of the LLM did not alter the scientific content or its interpretation.
19


Reviews:
[{"review_id": "1AH5u8urmF", "reviewer": "Reviewer_7JNX", "summary": "This paper proposes CurvSeg, a novel framework for skeleton-based temporal action segmentation (STAS) that addresses the long-standing tension between action classification and boundary localization. The key insight is geometric: well-separated classification features naturally induce high curvature within action segments and low curvature at transitions — forming a \"valley\" that serves as a strong prior for boundary detection.\nTo exploit this, the authors introduce two core components:\nCurvature-Guided Synergy (CGS): A bidirectional consistency mechanism where classification feature curvature guides boundary prediction, while boundary supervision regularizes classification features to enhance cluster compactness.\nExpert-Driven Decoupling (EDD): A Mixture-of-Experts module with task-specific experts that refine shared encoder outputs into adaptive representations for classification and localization.\nExtensive experiments on four benchmarks (PKU-MMD, LARa, MCFS-22/130) show consistent improvements over state-of-the-art methods, particularly in segmental F1 scores, validating the effectiveness of curvature-guided collaboration.\nThe work makes a compelling case for geometric priors in structured prediction tasks, offering both conceptual novelty and practical gains.", "questions": "Q1: In Section 3.3, you mention that low-curvature regions correspond to boundaries. But in Fig. 4(a), we see high curvature at boundaries. Could you clarify this apparent contradiction? Is it possible that both high and low curvature can indicate transitions depending on context?\nThis could change my understanding of whether curvature acts as a direct boundary detector or only an indirect regularizer.\n\nQ2: You show in Table 5 that curvature outperforms Euclidean/Cosine distance metrics. Have you considered comparing against learned boundary detectors (e.g., gradient-based saliency maps)? Does curvature still dominate in such comparisons?\n\nQ3: What happens if you apply the curvature signal only during training but remove it at inference? Would performance drop significantly? This would help quantify how much of the gain comes from architectural synergy vs. test-time guidance.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Originality\n\nInnovative use of representation geometry: Leveraging trajectory curvature as a cross-task signal is conceptually fresh and theoretically grounded (Appendix B). This moves beyond typical attention or fusion mechanisms.\nBidirectional synergy design: Unlike prior decoupled frameworks that treat tasks independently, CurvSeg establishes a mutual reinforcement loop, which is rare in STAS literature.\nTask-adaptive MoE without parameter explosion: The Gaussian expert routing is lightweight yet effective, enabling dynamic feature specialization without full dual-path architectures.\n\n Quality\n\nRigorous experimental evaluation: Results across four datasets, including ablation studies and comparisons with strong baselines (DeST, LaSA), demonstrate robust performance gains.\nWell-designed ablations: Tables 3–6 clearly isolate contributions of CGS and EDD, showing their individual and synergistic effects.\nQualitative visualization: Figure 4 effectively illustrates improved boundary precision and reduced over-segmentation.\n\n Clarity\n\nThe paper is well-written and logically structured, with intuitive figures (Fig. 1–2) explaining the core ideas.\nEquations are clearly presented, and Algorithm 1/2 provide sufficient implementation details.\nAppendices offer valuable theoretical justification and hyperparameter analysis.\n\nSignificance\n\nAddresses a fundamental limitation in STAS: insufficient cross-task collaboration despite semantic interdependence.\nDemonstrates that geometric structure in learned representations can be exploited for downstream tasks — an idea potentially applicable to other time-series problems (e.g., speech segmentation, medical signal analysis).\nOffers a new paradigm: using internal model dynamics (curvature) as supervisory signals, reducing reliance on external priors.", "weaknesses": "1) Limited discussion on failure cases\nWhile the method performs well overall, there is no analysis of when or why curvature fails as a boundary proxy. For example:\nIn gradual transitions (e.g., slow hand movement), curvature may not form clear valleys.\nNoisy skeleton data might amplify spurious curvature peaks.\nA brief error analysis (e.g., per-action performance drop) would strengthen the claims.\n\n2) Assumption of uniform segment partitioning\nThe EDD module divides videos into fixed-length segments (e.g., M=64), regardless of actual action duration. This could misalign temporal patterns for very short or long actions. Some discussion on adaptivity (e.g., content-aware segmentation) would improve robustness.\n\n3) Dependency on classification quality\nThe CGS module assumes that classification features already form compact clusters. If initial clustering is poor (e.g., due to ambiguous actions), curvature may not emerge reliably. The paper lacks sensitivity analysis under weak classification regimes.\n\n 4) Reproducibility concerns\nAlthough code will be released, some implementation details are missing:\nHow exactly are classification features $F_{cls}$ extracted? From the encoder output or after the classification head?\nIs the curvature computed per-joint or globally?\nThese should be clarified in the final version.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yRNpUw02Bt", "reviewer": "Reviewer_hoUk", "summary": "This paper addresses a core challenge in Skeleton-based Temporal Action Segmentation (STAS): the conflicting feature requirements for its two main sub-tasks, action classification (which needs temporal invariance) and boundary localization (which needs temporal sensitivity). Existing methods typically decouple these tasks, which prevents beneficial cross-task collaboration and creates \"information silos\". The paper proposes CurvSeg, a novel approach that synergizes these tasks using a geometric curvature guidance mechanism. The key insight is that in a well-learned classification feature space, the trajectory of skeleton frame features exhibits high curvature within an action segment (to stay within its class cluster) but low curvature at transitions when moving between clusters.", "questions": "See above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The paper introduces a novel curvature-based task synergy mechanism (CGS) that effectively exploits the geometric properties of feature sequences. This mechanism establishes a self-reinforcing loop where improved boundary detection and more discriminative classification features mutually enhance one another.\n\n2. The method is validated through comprehensive experiments on multiple benchmark datasets (PKU-MMD, LARa, MCFS-22, and MCFS-130) , where it achieves superior, state-of-the-art results. The most significant gains are seen in segmental F1 scores, directly validating the method's ability to enhance temporal boundary precision.\n\n3. Thorough ablation studies demonstrate the efficacy of each core component. The studies show that both the Expert-Driven Decoupling (EDD) and the Curvature-Guided Synergy (CGS)  independently improve performance. When combined, the full model achieves a synergistic effect, with performance gains surpassing the sum of the individual modules.\n\n4. The paper demonstrates that curvature is a more robust proxy for action boundaries than traditional distance metrics like Euclidean or Cosine. Curvature is sensitive to changes in the direction of the feature trajectory, making it better at detecting both gradual and abrupt action transitions.", "weaknesses": "1. The related works in Skeleton-based Temporal Action Segmentation are not fully discussed. Only two works in 2020 are discussed. More recent works shuold be incorporated.\n2. As reflected by Eq.9, the information of the classification head and localization head just interact once, not in a self-reinforcing loop as the authors describe.\n3. The description of the foundation framework in sec.3.2 owns too much space. As the baseline and foundational model, it should be compactly introduced.\n4. The key innovation is directly borrowed from the previous work (Shinet al., 2024), and adopted for STAS with a simple transfer.\n5. Although the paper asserts that high curvature corresponds to intra-segment motion and low curvature to boundaries, the theoretical link is only qualitatively motivated and relies on assumptions (e.g., class clusters as hyperspheres). The “Appendix B proof” simplifies dynamics to random walks within spheres, which is too idealized for real, noisy skeleton trajectories.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WyIoJruv7c", "reviewer": "Reviewer_t1pj", "summary": "The paper tackles skeleton-based action understanding by arguing that the geometry of feature sequences over time can be exploited to make classification and temporal localization help each other. The motivation is clear: current methods still show “insufficient cross-task collaboration,” so the authors introduce a curvature-based task synergy mechanism that uses geometric properties of feature trajectories to link both sub-tasks, and they couple it with a dual-expert weighting mechanism in a Mixture-of-Experts setup to adapt features to the task. As the authors themselves note, the approach still struggles with very noisy skeletons and complex multi-person cases, so there is room to improve robustness.", "questions": "1. Notations and definitions. Can you clarify the symbols in the method section (in particular F_s, F_{st}, F_{ST}, F_{\\text{gcn}}, and \\alpha_i, \\alpha_j) and make them consistent with Fig. 2?\n2. Figure 2 clarify.\n3. Rewrite Ablation on PKU-MMD (X-view): Why does the CGS-only setup outperform the full version for Edit and F1@10?\n4. Triplet/window construction: When you say “three consecutive points” but use t-w, t, t+w, what happens for w>1? Are intermediate points used and how are they aggregated into the final task representation?\n5. Can you provide a small visualization to support the high-curvature vs low-curvature claim, and explain the extra peaks in Fig. 4 (b)–(c) where the action does not change?\n \nFor more details, please see the Weakness section above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "Clear motivation.  The paper starts from an identifiable gap, weak collaboration between classification and localization, and address it directly.\n\nGeometric novelty. using curvature of feature sequences as a bridge between sub-tasks is a fresh angle compared to standard temporal smoothing or boundary refinement.\n\nTask-adaptive extraction: the dual-expert / MoE part makes the framework flexible to different task demands instead of using a single shared feature space.", "weaknesses": "Major 1. Across the method section, several symbols are introduced without being defined first, or they are not mathematically specified in a precise way. For example:\n\n\t•\tL176: \\alpha_i and \\alpha_j are not defined.\n\n\t•\tL182 – F_s is introduced without stating whether it is the same as the feature sequence in the problem formulation (X) or already an encoded representation.\n\n\t•\tEq. (4) and L191 – F_{ST} is used without an earlier, explicit definition. From context, it seems to be the spatio-temporal feature output of the Liformer / GCN stage, possibly the same as F_{\\text{gcn}}. Please unify the notation and add a one-sentence definition before Eq. (4).\n\n\t•\tL223 – “frame-wise classification features.” The text seems to refer to the same tensor that in Fig. 2 is denoted by the decoder classification head Y_{cl}. Please make these two references consistent.\n\n\t•\tL225–239 – windowed triplets. The text says “three consecutive points,” but the notation is F_{(cls, t-w)}, F_{(cls, t)}, F_{(cls, t+w)}. This is only strictly consecutive when w=1. For w>1, it is unclear: whether intermediate points are also used,  whether triplets are processed independently or aggregated, and how the final task representation is formed from them.\n\nPlease provide explicit sampling and aggregation equations.\n\n\t•\tL262 – F_{st} is mentioned again, but it is not clear whether this is the same as F_s, F_{st}, or F_{ST}. The shape is given as V \\times T without channels, which is hard to reconcile with the rest of the model.\n\n⸻\n\nMajor 2.\nIn the ablation results, two entries on PKU-MMD (X-view) — the Edit score and F1@10 — should be bold for the CGS-only configuration. I would like to know why this could be happening, because it would somehow break the idea that “EDD provides the high-quality.”\n\n\nTo make this section coherent:\n\t1.\tBold those two CGS-only numbers.\n\n\t2.\tAdd a short interpretation, e.g. that CGS might be better aligned with cross-view variability in PKU-MMD than the EDD augmentation in that specific setup.\n\n⸻\n\nMajor 3.\n\t•\tFigure 2 seems to have two different flow directions: the central pipeline is read from bottom to top, while the side modules are read from top to bottom. This makes it hard to know where the computation actually starts and where it ends.\n\n\t•\tVariable names are placed on top of the drawings and are hard to read (small font, low contrast, and sometimes overlapping the boxes).\n\n\t•\tThe figure repeats equations that are already in the text — equations (7) and (8) and L239 — which makes it redundant and not visually explanatory.\n\n\t•\tThe gray vertical line in the first column is not explained . if it is an encoder/decoder split, please state that.\n\n\t•\tIn text (L293) you say the feature is divided into M segments, but the figure shows g_1, g_2, g_3, which suggests a fixed number of Gaussians G=3. Please make the figure consistent with the text.\n\n\t•\tThe final step after the Gaussian generator ends in something like F_{m1}^{ST}, but it is not shown how these features are merged back into the task representation — that is precisely what readers will want to see.\n\n⸻\n\nMinor issues\n\n\t1.\tL218–220: the statement “intra-segment points must frequently change direction to remain within their class-specific boundary, resulting in high curvature, while inter-segment points exhibit low curvature as they move between class regions” — where is this shown to be true? It would be necessary to include a visualization or some evidence where this can actually be seen.\n\n\t2.\tL143 – “recent advances” needs citations. If you refer to recent advances, please add representative works.\n\n\t3.\tSome of the dimension strings in the description are of the form “(V \\times K) D \\times T \\times v” or “1 \\times I \\times K \\, D \\times D,” which can be a bit confusing regarding how they are multiplied.\n\n\t4.     In the hyperparameter analysis section, the parameter w is not discussed.\n\n\t5.\tSome sentences are not well expressed or well written. For example:\n\t•\tL216: “…feature space. (This observation can be formally proven: the average curvature of a random walk is inversely proportional to the radius of its bounding hyper-sphere. See Appendix B.) This…”\n\t•\tL448: a sentence should not start with a variable.\n\t•\tThe last sentence of L454. Please revise in general.\n\n\t6.\tFigure 4 – confusing “spins” in parts (b) and (c).\nIn Fig. 4 (b) and (c) there appear to be many sharp peaks / spins in regions where, according to the action annotation, the action does not change. Please explain why the method produces such high-frequency responses in stable segments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gqauYRdCeL", "reviewer": "Reviewer_doPp", "summary": "This paper addresses the challenge of conflicting feature requirements in skeleton-based temporal action segmentation (STAS). The key innovation is utilizing the curvature properties of well-learned classification representations on skeleton sequences, with high curvature within action segments and low curvature at transitions serving as geometric priors for boundary detection.", "questions": "Although the writing in this paper is relatively standardized and the method exhibits some innovations, the task of skeleton-based temporal action segmentation is limited. \n\nThis paper does not provide me with meaningful insights, and even if it were accepted, its contribution to the field would be quite limited. I suggest that the authors expand their method for more tasks to enhance its generality.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This paper proposes a novel curvature-based approach that leverages the geometric properties of feature sequences to foster effective collaboration between classification and localization sub-tasks.\n\nThe introduction of a dual-expert weighting mechanism within a Mixture of Experts (MoE) framework enhances the performance of the synergy mechanism by separately capturing semantic representations for classification and fine-grained temporal details for localization.", "weaknesses": "There are the following issues with Figure 1: (a) and (b) represent different STAS pipelines, while (c) and (d) do not; (c) and (d) are jumbled together without any spacing to distinguish them; and text in this figure is too small.\n\nLine 231, where, θt ∈ [0, π] quantifies. , should be deleted.\n\nLine 146, Task Decoupling in STAS  . is missing. \n\nThere is no analysis of the method's time complexity and runtime. Although a 1.5% improvement is achieved on the PKU-MMD dataset, it would be meaningless if it comes at the cost of increasing the model's runtime.\n\nIn temporal action segmentation, are the evaluation metrics Acc, Edit, and F1 reasonable? Why not use evaluation metrics similar to IoU?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)