Beyond Euclidean: Dual-Space Representation Learning for Weakly Supervised Video Violence Detection
Authors: Jiaxu Leng
Zhanjie Wu
Mingpi Tan
Yiran Liu
Ji Gan
Haosheng Chen
Xinbo Gao
Venue: NeurIPS 2024 poster (2024)
Keywords: weakly supervised video violence detection
ambiguous violence
hyperbolic graph convolutional network
dual-space interaction
Primary area: machine_vision

Abstract: While numerous Video Violence Detection (VVD) methods have focused on representation learning in Euclidean space, they struggle to learn sufficiently discriminative features, leading to weaknesses in recognizing normal events that are visually similar to violent events (i.e., ambiguous violence). In contrast, hyperbolic representation learning, renowned for its ability to model hierarchical and complex relationships between events, has the potential to amplify the discrimination between visually similar events. Inspired by these, we develop a novel Dual-Space Representation Learning (DSRL) method for weakly supervised VVD to utilize the strength of both Euclidean and hyperbolic geometries, capturing the visual features of events while also exploring the intrinsic relations between events, thereby enhancing the discriminative capacity of the features. DSRL employs a novel information aggregation strategy to progressively learn event context in hyperbolic spaces, which selects aggregation nodes through layer-sensitive hyperbolic association degrees constrained by hyperbolic Dirichlet energy. Furthermore, DSRL attempts to break the cyber-balkanization of different spaces, utilizing cross-space attention to facilitate information interactions between Euclidean and hyperbolic space to capture better discriminative features for final violence detection.  Comprehensive experiments demonstrate the effectiveness of our proposed DSRL.

Beyond Euclidean: Dual-Space Representation Learning for Weakly Supervised Video Violence Detection
Jiaxu Leng Zhanjie Wuz.2, Mingpi Tant.2 Yiran Liu', Ji Gan
Haosheng Chen Xinbo Gao 1.2
Chongqing University of Posts and Telecommunications, Chongqing, China
2 Chongqing Institute for Brain and Intelligence, Guangyang Bay Laboratory, Chongging, China gaoxbecqupt. edu.cn
Abstract
While numerous Video Violence Detection (VVD) methods have focused rep- resentation learning in Euclidean space, they struggle to learn sufficiently dis- criminative features, leading to weaknesses in recognizing normal events that visually similar to violent events (i.e., ambiguous violence). In contrast, hyperbolic representation learning, renowned for its ability to model hicrarchical and complex relationships between events, has the potential to amplify the discrimination be- tween visually similar events. Inspired by these, we develop a novel Dual-Space Representation Learning (DSRL) method for weakly supervised VVD to utilize the strength of both Euclidean and hyperbolic geometries, capturing the visual fea- tures of events while also exploring the intrinsic relations between events, thereby enhancing the discriminative capacity of the features. DSRL employs novel information aggregation strategy progressively learn event context in hyperbolic spaces, which selects aggregation nodes through layer-sensitive hyperbolic asso- ciation degrees constrained by hyperbolic Dirichlet energy. Furthermore, DSRL attempts to break the byber-balkanization of different spaces, utilizing cross-space attention facilitate information interactions between Euclidean and hyperbolic space to capture better discriminative features for final violence detection. Com- prehensive experiments demonstrate the effectiveness of our proposed DSRL.
Introduction
Compared with manually checking out violent events surveillance videos which is cime-consuming and laborious, Video Violence Detection (VVD), which focuses on identifying violent events in videos and provides automatic and instantaneous responses, has gained significant research attention due to its potential applications. However, it is expensive to annotate each frame in a video so that we can train VVD model with supervised learning. To address this, current methods often utilize weakly supervised settings to formulate the problem as multiple-instance learning (MIL)[T) task. These methods treat video as a bag of instances (i.e., snippets or segments), and predict their labels based on the video-level annotations.
According to the modality type of the input data, existing weakly supervised VVD methods can be roughly divided into two categories, unimodal with only vision input and multimodal with vision and audio input. The unimodal methods [29 36 g 10 20 learn the different distributions of normal and violent events through video-level labels, focus on finding valuable visual cues that are distinct from non-violent events and use them detect violent events, i.e., fighting. However, relying on visual cues
"Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

hierarchy of categories
Event
Space
Hyperbalic Metric
Viulent
Normal
visual
Ieee
Abuna
Fiatring
Playing bockey
Other
features
Dual Space
DSRL
enhance discrimination
The hierariby of event dev develogmen.
hierarchicul relation
Before the fighting During the lighting
Afler Bh fibhtng Afler the Raniing
Hypecbelic Space
Time
(a) Hicrarchical Diagram VVD
Superiority of DSRL for Ambiguous Violence Detection
Figure I: (a) Hierarchical diagram in Video Violence Detection (VVD). (b) Our DSRL enhances the detection of ambiguous violence by combining Euclidean and Hyperbolic spaces to balance visual feature expression and hierarchical event relations.
to identify violent events is sometimes unreliable, especially when facing visually ambiguous events, like normal physical collisions and fighting behavior in hockey games. To alleviate this problem, Wu et al [34] released a large multimodal violence dataset, named XD-Vin Vo-Viilenee, and accelcrated
series of multimodal VVD methods 34 35
37 23 Multimodal VVD methods incorporate
not only visual cues but also complementary audio information for improving the discrimination of violent events. Previous methods have employed Euclidean space representation learning and achieved good results in many other computer vision tasks 28) 32]. Despite advancements in audio-visual violent video detection (VVD) methods, their performance remains unsatisfactory in recognizing normal events that are visually similar to violent events (i.e., ambiguous violence) due to the limitations of Euclidean space. While visual features are fully extracted in Euclidean space, these methods fail to adequately capture and utilize the intrinsic relations between events.
To fully understand an event, the one hand, need explore the hierarchy of event categories, on the other hand, we need to sort through the events, including the trend before the event, the action during the event and the behaviour after the event, which reflects the hierarchy of event development, shown in Figure [a). An ambiguous violent event is confusing at the current category level or happening moment, but may easily be detected if we can grasp the hierarchical relations. Fortunately, hyperbolic representation learning, characterized by exponentially increasing the metric distances and naturally reflects the hierarchical structure of data, has gained attention and shown promising performance in computer vision tasks, like semantic segmentation [.. medical image recognition action recognition [24] [7], anomaly recognition [12] At present, only one method, HyperVD makes a preliminary attempt on the VVD task via hyperbolic representation learning. HyperVD introduces the Hyperbolic Graph Convolutional Networks (HGCN) (3). an extended version of Euclidean graph convolution for representation learning in hyperbolic space, to learn discriminative representations. However, HGCN employs a hard node selection strategy during the message passing, where the nodes whose correlation is higher than the threshold (a manual parameter) are selected for message aggregation and otherwise discarded, which leads insufficient hierarchical relation learning. In addition, existing VVD methods deploy feature embedding either in Euclidean or hyperbolic spaces. Representation learning in single space like picking the sesame and losing the watermelon, where the feature embedding is insufficient to guarantee the performance of VVD. On the one hand, hyperbolic representation Icarning strengthens the hierarchical relation of events but weakens the expression of visual features, the other hand, Euclidean representations emphasize visual features but ignore relations between events. Therefore, leveraging the advantages of both spaces is essential for improving the performance of VVD methods, shown in Figure lbb.
38
25
In this paper, we propose novel Dual-Space Representation Learning (DSRL) method for weakly supervised VVD under the multimodal input setting. Specifically, we designed two customized modules, the Hyperbolic Energy-constrained Graph Convolutional Network module (HE-GCN) and the Dual-Space Interaction module (DSI). Instead of adopting the hard node selection strategy in HGCN, HE-GCN selects nodes for message aggregation by our introduced layer-sensitive hyperbolic association degrees, which are dynamic thresholds determined by the message aggregation degree at

---

each layer. To better align with the characteristics of hyperbolic spaces, we introduce the hyperbolic Dirichlet energy to quantify the extent of message aggregation. Benefiting from the dynamic threshold, the layer-by-layer focused message passing strategy adopted by HE-GCN not only ensures the efficiency of information excavation but also improves the model's 's comprehensive understanding of the events, thus enhancing the model's 's ability to discriminate ambiguous violent events. Although hyperbolic representation learning enhances the understanding of hierarchical relations of events, the role of visual representations in violence detection cannot be discarded. However, fusing representation in different spaces remains a challenge, to break the information cocoon, DSI utilises cross-space attention to facilitate information interactions between Euclidean and hyperbolic space to capture better discriminative features, where Euclidean representations have effectiveness on the significant motion and shape changes in the video. while hyperbolic representations accelerate the comprehension of hierarchical relations between events, working together to improve the performance of violence detection in videos.
Contributions: (1) To the best of our knowledge, our DSRL the first method to integrate Euclidean and hyperbolic geometries for VVD, significantly improving discrimination of ambiguous violence and achieving state-of-the-art performance on the XD-Violence dataset in both unimodal and mul- timodal settings. (2) To better capture the hierarchical context of events, we design the HE-GCN module with a novel message aggregation strategy. where the node selection threshold is dynamic not fixed and determined by layer-sensitive hyperbolic association degrees based hyperbolic Dirichlet energy. (3) To break the information cocoon for better dual-space cooperation, visual discrimination from Euclidean and event hierarchical discrimination from hyperbolic. we propose the DSI module, which utilizes cross-space attention to facilitate information interactions.
2
Related Work
Weakly Supervised Video Violence Detection. Weakly supervised VVD requires identifying violent snippets under video-level labels, where the MIL [19] framework is widely used for denoising irrelevant information. Recently, progress has been made in weakly supervised VVD, with approaches categorized into two main categories: assionbbaedd and
sittoovviuall-bssee methods. Employing exclusively visual cues, vision- VVD endeavours to discern the occurrence of violent events
within videos. Most existing works 7 26 30 33 consider VVD as solely visual task, and CNN- based networks are utilized to encode visual features. However, these approaches overlook the interaction between different modalities and the relevant audio information, which could negatively impact the accuracy of violence detection. To alleviate this problem. Wu et al 34 released a large multimodal violence dataset, named XD- -Vollenee and accelerated a series of multimodal VVD methods [34, 35| [2] 37 23 18, 225. In contrast to unimodal methods, they incorporate not only visual cues but also complementary audio information for improving the discrimination of ambiguous violent events. Subsequently, many studies [22] 23 have focused on the integration of visual and audio information. There is also [or focused on solving the modality asynchrony problem. Despite the progress of weakly supervised multimodal VVD methods, all the above methods carry out representation learning in Euclidean Spaces, making it difficult effectively handle ambiguous violence.
Hyperbolic Representation Learning. Hyperbolic representation learning, characterized by ex- ponentially increasing the metric distances and naturally reflects the hierarchical structure of data, has gained attention and shown promising performance in computer vision tasks, like semantic seg- mentation [.. visual representation learning [9]. medical image recognition (38), action recognition [7] anomaly recognition [2]. More Recently, HyperVD [5] has made an initial attempt at the VVD task using hyperbolic representation learning. HyperVD incorporates Hyperbolic Graph Convolutional Networks (HGCN) (3 to acquire discriminative representations. However, while hyperbolic representation learning enhances the hierarchical relationships of events, it diminishes the representation of visual features. Therefore, we propose a novel Dual-Space Representation Learning to joint the strengths of Euclidean and hyperbolic space. Meanwhile, we design HE-GCN to gradually shift its focus from capturing global contextual information to concentrating on crucial detailed features, progressively capturing the hierarchical context of events.
3

---

3 Preliminaries
Given an video sequence S (S+) with non-overlapping segments. For a video segment, the weakly supervised VVD requires distinguishing whether it contains violent events via an events relevance label y/ E {0, 1), where y/ means in the current segment includes violent cues.
Problem Definition.
Hyperbolic Geometry. A Riemannian manifold (M. g) of dimension n a real and smooth manifold equipped with an inner product on tangent space In: T.M x TaM + R at each point E M, where the tangent space M n-dimensional vector space and can be seen as first-order local approximation of M around point In particular, hyperbolic space :gggg) ) constant negative curvature Riemannian manifold, is defined by the manifold DD= {ar E R2 c ||ar <
equipped with the following Riemannian metric:g. 22 where Xe 1-<||=|? and g In the Euclidean metric tensor. Considering the numerical stability and calculation simplicity of its exponential and logarithmic maps and distance functions, we select the Lorentz model [2] as the framework cornerstone.
Lorentz Model. Formally, an n-dimensional Lorentz model is the Riemannian manifold T (C", ge ). K is the constant negative curvature, o. diag (-1, 1...,, .., ., 1) is the Riemannian metric tensor. We denote L" as the n-dimensional hyperboloid manifold with constant negative curvature K:
L
x x
(x,x)
K xo 0
Letx,y E R"+1, then the Lorentzian scalar product is defined as:
(x. y) == -0000 + Σ J.lle
(2)
where L" is the upper sheet of hyperboloid in an (n+ dimensional Minkowski space with the origin (V-I/K, ...,, ..0. For simplicity, we denote point x in the Lorentz model as x E Lix
Tangent Space. Given the tangent space is defined an n-dimensional vector space approximat- ing Lx around v,
T = {y E Ra+1 x)xx)) x)xxx==0] =
Note that TuL is Euclidean subspace of TR++1, Particularly, we denote the tangent space at the origin as Toll..- Logarithmic and Exponential Maps. The mapping between hyperbolic spaces and tangent spaces can be done by exponential map and logarithmic map. The exponential map a map from a subset of a tangent space of Lg (i.e., TxLx) to L3 itself. The logarithmic map is the reverse map that maps back to the tangent space. For points x, E L E TxL such that v j and x + y, the exponential map expx (.) and logarithmic map logx (.) are given as follows:
xxp (v) cosh( (aah(v-K' K || sinh(v [Ivlle) V-K lille
(4)
y K (x,y)
log K (y) = d yx,y)
(5)
|ly K (x, cll
where/lv|| / v) denotes Lorentzian norm of and d (-.-) denotes the Lorentzian intrinsic distance function between two points x, y E Lik which is given as:
di (x.y) = arcosh (K (x.y)
(6)
4 Methodology
In this paper, we propose the Dual-Space Representation Learning (DSRL) method to improve the discrimination of ambiguous violence. Within DSRL, we first design the Hyperbolic Energy- constrained Graph Convolutional Network Module (HE-GCN) to better capture the hierarchical context of events (Sec. 4.1) ). Then, the Dual-Space Interaction Module (DSI) is introduced to break
the information cocoon for better dual-space cooperation (Sec. 4.2 An illustration of the main components of DSRL is provided in Figure

---

I Fosture Prepreceningg
Ryperbalic Energ-cowmaheed GON
Dusl-pacce Inseraction
Holonce Score
Ey
LSHAD
Rypetel Duccke
Arselation degiee
Exprecalla Mappiigg
Video
x
LOWAD construction
SCE
Lass
Apprertion
Ex
Vidkeae
GCN
Hoperbolk
(nDe)
Leger Sensitive Pyyerkaki (SHAD)
Voaal
Aedle
Aw?
dighes
Nadtpp
CSEAAY 6.A, <aRAA
Figure 2: conceptual diagram of our DSRL.
4.1 Hyperbolic
Graph Convolutional Network Module (HE-GCN)
HE-GCN primarily involves mapping features from Euclidean space to hyperbolic space, then transforming the features, constructing the message graph by calculating hyperbolic Dirichlet energy and Layer-Sensitive Hyperbolic Association Degree, and finally aggregating messages obtain the new feature graph.
Mapping from Euclidean to hyperbolic spaces. Let { Jiev be input Euclidean node features, and
o = [1, ., 0] denote the origin on the manifold the Lorenzt model. There is (o, [0, xf ))=
0. where (,, ) c denotes the Lorentz inner product defined in Eq. 2 We can reasonably regard [0 x E
as node on the tangent space at the origin o. HE-GCN uses the exponential map defined in Eq. to generate hyperbolic node representations on the Lorentz model:
of = expo (10, (00,2,1) cosh (1<=111) sinh (10,1122
Hyperbolic Feature Transformation. According to [5]. we reformalize the lorentz linear layer to w E 12++1, w mmx((+1) satisfying Vx E L", (M(M)x E L", where
learn matrix M
fx (++1) + R(m+1)x(n+1) should be a function that maps any matrix to a suitable one for the hyperbolic linear layer. Specifically, Vx E L M E (m++1) fxM) is given as
[ [w=)=
//Wx|2-17
((M) = fx
Theorem Vr E L", M ()+++))(++1), we have (M)x E LR-
For simplicity, morel general formula of hyperbolic linear layer for feature transformation x with activation, dropout, bias and normalization.
y
based on fx ww]
y == (x) = V 11 (Wx, v)] -1/K &Wxx
where & E Lg. W E (=+1+++), E R=+1 denotes a velocity (ratio to the speed of light) in the Lorentz transformations, and Ø an operation function: for the dropout, the function is Φ (Wx, v)
xx(66x+66)
Wdropout (x); for the activation and normalization (Wx, v) = [Wi
(Wh (x) + b). where is the sigmoid function, and b' are bias terms, À 0 controls the scaling range. h is the activation function. And then, we need to construct the message graph.
Hyperbolic Dirichlet Energy. Given the hyperbolic embedings x {x; L 1 the hyperbolic Dirichlet energy (HDE) E (x) is defined as
(expx K
K (x:) log, VI +1+ dd
K log (x,)
KK (x)
Σ
di
expo
(10)
VI+ dj
1,j=1
5

---

where d: denotes the node degree of node i/j. The distance di (x.y) between two points x, E L the geodesic. Given that each node is connected to every other node videos, resulting in node degree di of n - (where n is the total number of nodes), the formula for hyperholic Dirichlet energy can be simplified.
N Σ fJ==1
E (x)
22 x,)x
(11)
2
HDE is used to measure the similarity between node features in order to gauge the degree of information aggregation among features. is evident that hyperbolic message aggregation progresses, the similarity between features gradually decreases. Hyperbolic message aggregation reduces HDE. It can be expressed as: ER (x(++)) < EB (x()), where I is the layer number. Layer-Sensitive Hyperbolic Association Degree. Based HDE, we design Layer-Sensitive Hyperbolic Association Degree (LSHAD) to guide the node selection strategy for our construction of graphs for message aggregation. It is defined as
LSHAD; = = (z) sigmoid(3k - Y EK (1)+1
(12)
where f-) is a function related is and E (x). le is the current layer number. | and y are hyperparameters.
Lorentzian similarity. Based on the Lorentzian distance, the Lorentzian similarity to measure the feature semantic similarity between nodes is given by
Ls (Zi, xz)) exp(- (I., 2;)),
(13)
where di (t,-) is the Lorentzian intrinsic distance function. We define the initial adjacent matrix A RXxT via lorentz similarity:
A = softmax(Lsstte x;)).
(14)
LSHAD Construct rules. With LSHAD, we propose message graph construction rules, called LSHAD construct rules. It is defined as
[ A.ji if A 2 LSHAD* (1 =0
(15)
if A < LSHAD*
where Ai, means the lorentz similarity between nodes and j in the graph G. Formally, it enforces the elements of the adjacency matrix that are less than LSHAD be zeros. Finally, we can dynamically construct message graphs at each layer via LSHAD construct rules to obtain better contextual information. Following LSHAD construction rules, we can dynamically construct message graphs G", which we then use to perform message aggregation operations to obtain better contextual information.
Hyperbolic Message Aggregation. We use graph G' to perform message aggregation, and the message aggregation can be defined as:
MA(y.)
(16)
E Ausalle
where m is the number of nodes. y; the node features.
4.2
Dual-Space Interaction Module
Although hyperbolic representation learning enhances understanding of event hierarchies, visual representations remain crucial in violence detection. Fusing representations from different spaces is challenging; thus, DSI employs cross-space attention to facilitate interactions between Euclidean and hyperbolic spaces.
Cross-Space Attention Mechanism. Cross-Space Attention Mechanism utilizes the Lorentzian metric to calculate attention scores between nodes from different spaces, accurately measuring semantic similarity and better preserving their true relationships by computing the nonlinear distance between them. We denote the features Fuclidean space as VE and the features in hyperbolic space VH. CSAg-A models the between-graph interaction and guides the transfer of inter-graph

---

message from VE to Vy. First, we use linear layer to transform VH to the key graph Vi and the value graph and VE to the query graph v. Then, use lorentzian metric calculate the attention map Ag-H as follows:
Iy) if Ls( 2z) if LZ, xz) S X,
AE-- soflmar (Ls(( V&)), Ls( (1))
0
(17)
where (-(-) is Lorentzian similarity defined in Eq. 13 and À is the threshold value to eliminate weak relations and strengthen correlations of more similar pairs. The representation from E to H can be formulated as follows:
N CSA VE) softmar AE x Ve
V..
(18)
Vd
The interaction process in DSI can be represented as follows:
V
Vn) + VE
V CSA VVg + Vи
(19)
VF = MaxPool + V/1)).
where V2 represents the features obtained by enhancing Euclidean space features using hyperbolic space features, and a controls the contribution of the enhanced features and Vg. Max Pool the max pooling operation and @ represents the concatenation operation. Learning Objective. We use binary cross-entropy as our classification loss. Its calculation formula is:
N
Loss = N
Σ (y) log(ū.) + (1 - y.)log(1 (.)),
(20)
where is true label, is is the predicted label, N is the batch size.
5 Experiments
5.1 Experiments Setup
conduct experiments on ID-Violence, which is the only and extremely challenging VVD dataset with multimodal information. Under the unimodal input setting, both the XD- Vi-Vencee 34 and UCF-Crime 30 datasets are used to evaluate our method. More details of the two dataset settings are provided in the Appendix. Evaluation Metrics. To quantitatively evaluate the performance, we follow standard pratice [4 I8 6). For XD-Violence, For UCF-Crime, we adopt the area under the curve the frame-level receiver operating characteristic (AUC) to evaluate performance.
Datasets.
Under the multimodal input setting, we follow 34 37 25
utilize the frame-level average precision (AP) as the evaluation metric.
5.2 Comparisons with State-of-the-art Methods
In T with multimodal input, our DSRL demonstrates superior performance on XD-Vio FD-Violence, surpassing the best method that uses Euclidean space representation by 4.21% 1 and that uses only hyperbolic space representation by 1.94%. These results highlight the effectiveness of DSRL in learning discriminative features and prove that our dual-space representation learning integrates the benefits of Euclidean and hyperbolic space well. From Table T it also can be found that DSRL achieves SOTA performance under unimodal input settings on ID-Violence, outperforming existing state-of-ther art methods. Furthermore, to analyse the generalization of our method, we also report the performance the UCF-Crime dataset, achieving an accuracy of 86.38%, comparable to current state-of-thc-art methods.
5.3 Ablation Studies
We conduct ablation studies on various design choices of our DSRL to demonstrate their contributions to the final results in Table
7

---

Table 1: Comparisons of frame-level AP performance on AD-Violence and AUC performance on
UCFF-Crie datasets under different input settings. UCF-Crime only has visual modality input.
Methods
Input Setting
Feature Space
UCF-Crime
XiD-leneence
Sultani al. [30]
Unimodal
Euclidean
76.21
73.20
Wu et al. [33
Unimodal
Euclidean
82.44
75.90
RTFM 31
Unimodal
Euclidean
84.30
77.81
MSL 16
Unimodal
Euclidean
85.30
78.28
MGFN
Unimodal
Euclidean
86.98(1*t)
79.19 193rd))
UMIL [18
Unimodal
Euclidean
86.75(2nd)
81 31.66(2nd)
CU-Net 39
Unimodal
Euclidean
86.22
78.74
Ours
Unimodal
Euclidean and Hyperbolic
86.38(3rd)
82.01(1*t)
HL-Net [34]
Multimodal
Euclidean
78.64
Wu et al. 33
Multimodal
Euclidean
78.64
Pang et
23
Multimodal
Euclidean
79.37
UMIL [18
Multimodal
Euclidean
81.77
Zhang et al.
Multimodal
Euclidean
81,43
MACIL-SD
Multimodal
Euclidean
83,40(3rd)
HyperVD
2
Multimodal
Hyperbolic
85.67(2
Ours
Multimodal
Euclidean and Hyperbolic
87.61(1*t)
Table 2: Ablations XD-Violence dataset.
Euclidean
Hyperbolic
DSI
XD. Violence
GCN
HE-GCN
HGCN
Concat
Cosine Metric
Lorentzian Metric
Multimodal
Unimodal(56)
84.04
77.95
85.01
77.93
86.46
79.70
86.91
80.72
87.61
82.01
Component-wir ablations. To study the impact of each component in the DSRL, including HE-GCN and DSI, we start with a baseline model that only applies GCN and progressively adds each component. Table that the baseline yields inferior performance due learning representations only in Euclidean space (1*t row). Then, HE-GCN is employed and the representations from different spaces are simply concat at this stage, benefit from the hicrarchical context of events, resulting an improvement of 2.42% on multimodal setting and 1.75% on unimodal setting respectively (3*d row). Then DSI introduced to facilitate information interactions, which further enhances the performance on multimodal input setting by 1.15% and unimodal input setting by 2.31 % (5th row). 2) Effect of LSHAD. LSHAD is a crucial element of our HE-GCN for constructing the message graph. In Table 2nd and 3rd rows), we present experiments comparing HGCN used in HyperVD (25]. which employs a hard node selection strategy (nodes selected by fixed threshold), with HE-GCN, which uses our introduced layer-sensitive hyperbolic association degrees for node selection in message aggregation. The performance of AP improved by 1.45% in the multimodal setting and by 1.77% on the unimodal setting. These results demonstrate that our ISHAD is better strategy and helps to capture the hicrarchical context of events. 3) Lorentzian metric DSL. As shown Table 2 4th and 5tt rows), also explore the effect of cosine similarity and Lorentzian metric in our DSI, and the results show that using cosine similarity to calculate the attention between nodes resulted a 0.7% lower performance compared with using the Lorentzian metric. This indicates that the Lorentzian metric more effective in measuring feature similarity across different spaces.
5.4 Qualitative Results
1) Feature Discrimination Visualization. We present t-SNE visualizations of feature distributions on the XD-Violence test set. shown Figure 3 red dots represent violent features, while purple dots represent normal features. The clear clustering of violent and non-violent features demonstrates the effectiveness of DSRL.
2) Qualitative Visualizations. We illustrate the qualitative visualizations of VVD for the test video

---

Figure 3: t-SNE visualization of vanilla and DSRL features for the video on XD-Violence
14) Video Diclldd L905
11-0010000000 22-0
Vidoo BuCoo mootyyyl 2009 -00-
Man AED-0
DSRL
Bazeline
Ground Truth
frame of normallty
frame vialence
Figure 4: Frame-level scores and violence localization examples for the test video from DD-Violence dataset.
from the AD-Violence dataset. Figure shows that DSRL can accurately detect violent events and has a better detection performance than the baseline.
3) Qualitative Visualizations of DSRL in the Context of Ambiguous Violence. We put up qualita- tive visualizations of DSRL. when handling ambiguous violence. Figure[]deemnstrale that DSRL effectively resolves ambiguous violence, which single-space representation learning struggles with. In Figure S(a), the first frame shows smoke caused by fire. Euclidean space representation, relying on visual features, misidentifies the smoke as violence. Hyperbolic space representation, consider- ing contextual information, also misidentifies it due to preceding violent frames. DSRL, however, combines both perspectives: Euclidean space flags the smoke potential violence indicator, while hyperbolic space recognizes the fire context. This integration allows DSRL to accurately classify the smoke as non-violent. This supports our motivation: hyperbolic representation enhances hierarchical event relations but weakens visual feature expression, while Euclidean representation emphasizes visual features but overlooks event relationships. DSRL. effectively addresses ambiguous violence, which is challenging for either space alone. More performance visualizations are included in the Appendix.
Time
[2] Wideo Trless 2014
[4) Vicca Birs (ATTNN04
Hyperbolic
DSRL
Ambiguaus
Figure 5: Some visual results of DSRL in the context of ambiguous violence, "Euclidean" represents the results of GCN only. "Hyperbolic" refers to the results HyperVD.

---

5.5 Comparisons of Computing Resources and Training Time
Our designs, HE-GCN and DSI, are specifically crafted to effectively capture hierarchical contextual information of events and to integrate two distinct spaces of information, respectively. To evaluate the benefits of these designs, we conducted experiments on the XD-Violence dataset, training three models for 30 epochs each BaselineHHE-GCN, and Baselnn+HEEGGNNAL our DSRL model demonstrated a 3.57% improvement in Average Precision (AP) compared the Baseline. Additionally, the training time per epoch increased by only 41 seconds, and memory usage rose by just 4.1 GB. Both increases are within a reasonable range, making the performance gains achieved by DSRL well worth the additional resource consumption.
single NVIDIA RTX A6000 GPU: the Baseline (GCN), (referred to as DSRL), As shown in Table 3
Table 3: Comparison of computing resources and training time.
Methods Baseline(GCN)
Params
Training time per epoch
Truining time
Video memory usage
AP (%)
0.7734M
2min
60min
4,24GB
84,04
Baseline+HE-GGr
0.8975M
2min19s 2min41s
69min39s
7,03GB
86.46
0,9966M
80min 19s
8,34GB
87.61
5.6 Analysis of Model Computational Complexity and Speed
The computational efficiency of the DSRL model is crucial, especially for real-time applications. Our analysis confirms that the model meets the requirements for real-time processing, as demonstrated by the following results. Our experiments were conducted on a single NVIDIA RTX A6000 GPU, For video input, the model processes at a rate of 83.8 87 FPS, handling only video data. The model's parameters are relatively lightweight, totaling 13.4 MB, with I3D parameters 12.49 MB and DSRL parameters at 0.91 MB. This compact size ensures quick response times. When processing video and audio inputs. the model maintains high processing speed of 56.86 FPS, despite the added computational load from audio processing. The total parameter size for this configuration is 85.54 MB, comprising I3D parameters at 12.49 MB, VGGish parameters at 72.14 MB, and DSRL parameters at 0.91 MB. These results illustrate that our model exhibits excellent real-time performance, even with multimodal inputs, making it suitable for latency-sensitive real-world applications.
Conclusions
In this paper, we propose a comprehensive geometric representation learning method, Dual-Space Representation Learning (DSRL) which integrates the benefits of Euclidean and hyperbolic geome- tries to improve the discrimination of ambiguous violence. Hyperbolic Energy-constraine Graph Convolutional Network (HE-GCN) is designed to better capture the hicrarchical context of events. Additionally, Dual-Space Interaction (DSI) is designed to facilitate information interactions. Our method achieves SOTA performance on the XD- Violenee dataset in both unimodal and multimodal settings, especially excelling in resolving ambiguous violence.
Limitions. Our DSRL is effective for VVD in multimodal input setting. However, DSRL only utilizes basic audio information, potentially overlooking the more detailed semantic content present in the audio. How to further narrow this limitation is our future research focus. Due to the bias issues the XD- Violeneee and UCF-Crime datasets, which fail to adequately represent diverse backgrounds, the fairness and generalization of the model may be affected. In the future, we hope to use more diverse datasets and conduct bias analysis to prevent the model from producing unfair outcomes due to false associations with gender race.
Acknowledgements. This work was supported in part by the Science and Technology Inno- vation Key R&D Program of Chongqing under Grant No. SSTB202TIADD-STXOO by the National Natural Science Foundation of China under Grants No. 62472060, 62441601, U23A20318, and 62221005, part by the Natural Science Foundation of Chongqing under Grand No. CSTB2022NSCQ-MSS 1024, CSTB200NSSQQ11200061 in part by the Science and Technol- ogy Research Program of Chongqing Municipal Education Commission under Grant No. KJZD- K202300604, and in part by the Chongqing Institute for Brain and Intelligence.
in part
10

---

References
[1] Mina Ghadimi Atigh, Julian Schoep, Erman Acar, Nanne Van Noord, and Pascal Mettes. Hyperbolic image segmentation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 4453-4462 4462, 2022.
[2] Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? new model and the kineties dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299-6308, 2017.
[3] Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. Hyperbolic graph convolutional neural networks. Advances neural information processing systems, 32, 2019.
[4] Lin Chen. Topological structure in visual perception. Science, 218(4573))699 1982.
[5] Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou, Fully hyperbolic neural networks. arXiv preprint arXiv:2105. 14686, 2021.
[6] Yingxian Chen, Zhengzhe Liu, Baoheng Zhang, Wilton Fok, Xiaojuan Qi, and Yik-Chung Wu. Mgfn:
slance-and-focus network for weakly weakly-supervised video anomaly detection. Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 387-395, 2023.
[7] Jia-Chang Feng, Fa-Ting Hong, and Wei-Shi Zheng. Mist: Multiple instance self-training framework for video anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14009-14018, 2021.
[8] Yuan Gao, Hong Liu, Xiaohu Sun, Can Wang, and Yi Liu. Violence detection using oriented violent flows. Image and vision computing, 48:37-41, 2016.
[9] songwei Ge, Shlok Mishra, Simon Kornblith, Chun-Liang Li, and David Jacobs. Hyperbolic contrastive learning for visual representations beyond objects. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 6840-6849) 2023.
[10] Tal Hassner, Yossi Itcher, and Orit Kliper-Gross. Violent flows: Real-time detection of violent crowd behavior. In 2012 IEEE computer society conference on computer vision and pattern recognition workshops, pages 1-6. IEEE, 2012.
[11] Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Chan- ning Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al. Can architectures for large-scale audio classification. 2017 ieee international conference on acoustics, speech and signal processing (icassp), pages 131-135. IEEE, 2017.
[12] Jie Hong, Pengfei Fang, Weihao Li, Junlin Han, Lars Petersson, and Mehrtash Harandi. Curved geometric networks for visual anomaly recognition. IEEE Transactions on Neural Networks and Learning Systems, 2023.
[13] Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu, John E Hopcroft, and Kilian Weinberger. Snapshot ensembles: Train 1, get m for free. arXiv preprint
2017.
[14] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.69881 2014.
[15] Shuang Li, Fan Li, Jinxing Li, Huafeng Li, Bob Zhang, Dapeng Tao, and Xinbo Gao. Logical relation inference and multiview information interaction for domain adaptation person re- identification. JEEE Transactions on Neural Networks and Learning Systems, 2023.
[16] Shuo Li, Fang Liu, and Licheng Jiao. Self- Self-training multi-sequence learning with transformer for weakly supervised video anomaly detection. Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 1395-1403, 2022.
[17] Teng Long, Pascal Mettes, Heng Tao Shen, and Cees GM Snoek. Searching for actions on the hyperbole. Proceedings of the IEFF/CVF Conference on Computer Vision and Puttern Recognition, pages 1141-1150, 2020.

---

[18] Hui Lv. Zhongqi Yue, Qianru Sun, Bin Luo, Zhen Cui, and Hanwang Zhang. Unbiased multiple instance learning for weakly supervised video anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 0022-8031, 2023.
[19] Oded Maron and Tomás Lozano- Pérez. A framework for multiple- multiple-instance instance learning. Advances in neural information processing systems, 10, 1997.
[20] Sadegh Mohammadi, Alessandro Perina, Hamed Kiani, and Vittorio Murino. Angry crowds: Detecting violent events videos. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdum, The Netherlands, October 11-14, 2016, Proceedings, Part VII 14, pages 3-18. Springer, 2016.
[21] Maximilian Nickel and Douwe Kiela. Learning continuous hierarchies in the lorentz model of hyperbolic geometry. In International conference on machine learning, pages 3779-3788. PMLR, 2018.
[22] Wen-Feng Pang, Qian-Hua He, Yong-jian Hu, and Yan-Xiong Li. Violence detection in videos based on fusing visual and audio information. In ICASSP 2021-2021 IEEE international conference acouslics, speech and signal processing (ICASSP), pages 2260-2264. IEEE, 2021.
[23] Wenfeng Pang, Wei Xie, Qianhua He, Yanxiong Li, and Jichen Yang. Audiovisual dependency attention for violence detection videos. IEEE Transactions on Multimedia, 2022.
[24] Wei Peng, Jingang Shi, Zhaoqiang Xia, and Guoying Zhao. Mix dimension in poincaré geometry for 3d skeleton-based action recognition. In Proceedings of the 28th ACM International Conference on Multimedia, pages 1432-1440, 2020.
[25] Xiaogang Peng, Hao Wen, Yikai Luo, Xiao Zhou, Keyang Yu, Yigang Wang, and Zizhao Wu. Learning weakly supervised audio-visual violence detection in hyperbolic space. arXiv preprint arXiv:2305. 18797, 2023.
[26] Fernando J Segador,, Juan A Álvarez-García, Jose L Salazar-Gonzále and Tatiana Tommasi. Crimenet: Neural structured learning using vision transformer for violence detection. Neural networks, 161:318 1-2399998...11 2023.
[27] Jiangming Shi, Xiangbo Yin, Yeyun Chen, Yachao Zhang, Zhizhong Zhang, Yuan Xie, and Yanyun Qu. Multi-memory matching for unsupervised visible-infrared person re-identificatiin. In ECCV, page 456-4744 2024.
[28] Jiangming Shi, Yachao Zhang, Xiangbo Yin, Yuan Xie, Zhizhong Zhang, Jianping Fan, Zhongchao Shi, and Yanyun Qu. Dual pseudo-labels interactive self-training for semi-supervised visible-infrared person re-identification. In ICCV, pages 11218-11 1228, 2023.
[29] Swathikiran Sudhakaran and Oswald Lanz. Learning to detect violent videos using convolutional long short-term memory. In 2017 14th IEEE international conference on advanced video and signal based surveillance (AVSS), pages 1-6. IEEE, 2017.
[30] Waqas Sultani, Chen Chen, and Muharak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6479-648 6488, 2018.
[31] Yu Tian, Guansong Pang, Yuanhong Chen, Rajvinder Singh, Johan W Verjans, and Gustavo Carneiro. Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4975-4986, 2021.
[32] Yiming Wang, Guanqiu Qi, Shuang Li, Yi Chai, and Huafeng Li. Body part-level domain alignment for admain-adaptive person re-identification with transformer framework. IEEE Transactions on Information Forensics and Security, 17:3321-3334, 2022.
[33] Peng Wu and Jing Liu. Learning causal temporal relation and feature discrimination for anomaly detection. IEEE Transactions on Image Processing, 30:3513-3527, 2021.
12

---

[34] Peng Wu, Jing Liu, Yujia Shi, Yujia Sun, Fangtao Shao, Zhaoyang Wu, and Zhiwei Yang. Not only look, but also listen: Learning multimodal violence detection under weak supervision. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Purt XXX 16, pages 322-339. Springer, 2020.
[35] Peng Wu, Xiaotao Liu, and Jing Liu. Weakly supervised audio-visual violence detection. IEEE Transactions Multimedia, 25:1674-1685, 2023.
[36] Long Xu, Chen Gong, Jie Yang, Qiang Wu, and Lixiu Yao. Violent video detection based on mosift feature and sparse coding, In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3538-3542. IEEE, 2014.
[37] Jiashuo Yu, Jinyu Liu, Ying Cheng, Rui Feng, and Yuejie Zhang. Modality-awarr contrastive instance learning with self-distillation for In Proceedings of the 30th ACM International Conference on Multimedia, pages 6278 6278-6287,
audio visual violence detection.
2022.
[38] Zhen Yu, Toan Nguyen, Yaniv Gal, Lie Ju, Shekhar S Chandra, Lei Zhang, Paul Bonnington, Victoria Mar, Zhiyong Wang, and Zongyuan Ge. Skin lesion recognition with class-hierarchy regularized hyperbolic embeddings. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 594-603. Springer, 2022.
[39] Chen Zhang, Guorong Li, Yuankai Qi, Shuhui Wang, Laiyun Qing, Qingming Huang, and Ming- Hsuan Yang. Exploiting completeness and uncertainty of pseudo labels for weakly supervised video anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16271 161200, 2023.
13

---

Appendix
Here we provide the proof of Theorem in Sec. [ The Technical details of the DSRL are introduced in Sec. B Moreover, the datasets settings and implementation details are shown in Sec. C We
conduct additional experiments in Sec. D and more qualitative results in Sec. Broader impacts are listed in Fec
A Proof of Theorem.
Theorem 1. vx E IL, M R(m+1)x(++)) we have fa (M)x E LR.
Proof
VIWWIP--IRK v.
VIIWI|R- 1/K
y y= A(M)x
=
Wx
Wx
(1,8)22
-(1 - 1/K) + (Wx) WW
||Wx||2 + ||
K
Thus, (M)x lies on the manifold C the Lorentz model.
B Technical details of the DSRL
As depicted in Figure 2 the entire process be divided into feature preprocessing integrate the features of the two modalities; representation learning in hyperbolic space to learn the hierarchical context of events; representation learning Euclidean space to learn the visual features of events; interaction between the two spaces to promote cross-space enhancement; and finally, feeding the features into a hyperbolic classifier for classification.
Feature Preprocessing. Following the previous works 34 23), the visual and audio segments are processed by the I3D [2] network pretrained on the Kinetics-400 dataset and the VGGish network pretrained on a large You'll YouTube dataset, respectively. After that, we perform further feature extraction using simple convolution and pooling operations. A simple cross-mode moosl attention mechanism is then employed to enhance the audio features, which are subsequently concatenated with the visual features form the fused features.
Hyperbolic Representation Learning. this part, we first use HE-GCN to learn the hierarchical context of events. Meanwhile, the temporal relation is also crucial for numerous video-based tasks. Therefore, we construct a temporal relation graph directly based on the temporal structure of a video and learn the temporal relation among snippets in hyperbolic space via HGCN. Its adjacency matrix A E TxT is only dependent on temporal positions the i-th and j-th snippets, which can be defined as:
A?, exp(
(21)
where a controls the range of influence of distance relation. Finally, we use HE-GCN for semantic message aggregation and HGCN for temporal message aggregation, then concatenate them obtain a hyperbolic space representation.
Euclidean Representation learning. The process of representation learning in Euclidean space is similar to that in hyperbolic space and follows a dual-branch structure, considering both semantic and temporal relationships. We use cosine similarity to compute the semantic similarity in Euclidean space and use Eq. 21 to compute temporal relationships. Finally, we employ GCN for message aggregation and concatenate them obtain the final representation in Euclidean space.
Dual-Space Interaction. this part, we primarily use cross-space attention enhance the interaction between the features of the two spaces. The detailed process is described in Sec. 4.2
14

---

Hyperbolic Classifier. As shown in Figure 2 we input the enhanced embeddings from DSI into hyperbolic classifier utilizing Lorentzian metric, which can be formalized as:
S afe + W), 6)
(22)
where is sigmoid function and W is weight matrices. b and denotes bias term and hyper-parames respectively. Lastly, we supervise the training of the violence scores obtained by the model with the real labels using the binary cross-entropy loss.
C Experimental Details
Dataset. We conducted experiments on XD- AD-Violence with both multi modti- input settings and single- modal input settings. Additionally, we performed experiments on UCF-Crime with single-modal input settings to demonstrate the generalization capability of DSRL.
1) XD-Violenee dataset is by far the only available large-scale audio-visual dataset for violence detection, which is also the largest dataset compared with other unimodal datasets. XD- Violence consists of 4,757 untrimmed videos (217 hours) and six types of violent events, which are curated from real-life movies and in-the-wild scenes on YouTube. For XD-Violenee dataset, only video-level annotations are provided.
2) UCF-Crime dataset is large-scale dataset comprised of real-world videos captured by surveillance cameras. It consists 1,610 training videos annotated with video-level labels and 290 test videos annotated the frame level to facilitate performance evaluation. The videos are collected from different scenes and encompass 13 distinct categories of anomalies.
Implementation Details. The visual sample rate is set to 24 fps. and visual features extracted by a sliding window with a size of 16 frames. For the audio data, we first divide each audio into 960-ms overlapped segments and compute the log-mel spectrogram with 96 64 bins. Our proposed method trained for 30 epochs in total, and the batch size is 256. The initial learning rate 0.001, which dynamically adjusted by a cosine annealing scheduler [3] We use Adam [14] as the optimizer without weight decay. For hyper-paat syper-parameters, we set / as 0.8, 7 as 1.2, a as 0.3, and dropout rate as 0.6. Following [25], a is empirically set e. For the MIL, we the value & of k-max activation as T
16 1j. where T denotes the length of the input feature.
Experiments Compute Resources. We use an Intel(R) Xeon(R) Platinum 8260 CPU 2.40GHz,
NVIDIA RTX A6000 GPU to conduct experiments. We use CUDA 12.2, Python 3.9.16, and Pytorch 1.12.1.
D Additional Experiments
Reasons for the design choices in LSHAD with its multiple hyperparameters and threshold criteria.
Inspired by the Global-first principle 4 that humans always have cognition on global first and then focus on local, we propose a novel node selection strategy, which guarantees the model captures the broader global context first with relaxed threshold the beginning of message aggregation and then focuses on the local context with more strict thresholds. To achieve this, we introduce the LSHAD construction rule, which calculates an LSHAD threshold based on the number of the current layer K and hyperbolic Dirichlet energy of the current layer. As the K increases and the hyperbolic Dirichlet energy decreases, the LSHAD threshold increases and is limited to 0 and 1 by the sigmoid function. If there is no 3 and 7, the threshold in the first layer will be strict (> 0.5), causing the overlook of some global context information. Therefore, to make our node selection threshold conform to the Global-first principle, we introduced the two hyperparameters in LSHAD, where 3 controls the influence of the number current layer k and 7 acts as bias fine-tune the threshold. Moreover, we conducted an ablation study to determine the optimal value of the two hyperparameters (3.y). where 3 ranges from [0.2,0.4.0.6.00 8, 1.0] and y ranges from 11.0, 1. 1.4.1.6,1 1,1.6,1188,2.01 8,2.0]. The results in the table below reveal that when / is 0.4, the performance is optimal, so we chose pair (0.8,
1.2) from this set.
Ablation studies on some hyperparameters in DSL. Moreover, we conduct ablation studies on some hyperparameters DSI. Table and 6 show the experimental results.
15

---

Table Ablation studies on and ?.
1.0
1.2
1,4
1.6
1.8
2.0
3
0.2
85.22
87.12
86.32
86.60
86.31
86.86
0,4
87.52
85.22
87.12
86.32
86.60
86.31
0.6
87.61
87.52
85.22
87.12
86.32
86.60
0.8
87.29
87.61
87.52
85.22
87.12
86.32
1.0
86.29
87.29
87.61
87.52
85.22
87.12
Table 5: Ablation studies on À of DSL.
A AV-AP
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0,9
87.14
87.38
87.1
87.32
86.53
87.2
87.4
87.61
87.23
Table 6: Ablation studies of DSSL
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
AV-AP
87.18
87.16
87.61
87.21
87.26
86.91
87.42
86.59
87.51
E
Visualization
Inference visualizations of the ablation modules.
In this section, we have supplemented inference visualization results for the ablation module, and conducted corresponding visualisation experiments to explore the discriminative power of the model w/ or w/o our core modules, HE-GCN and DSI. We conducted the analysis from two dimensions (the feature-level and the frame-leve), and the results are shown Figure 6 and Figure 7 At the feature-level, the results are shown in Figure Compared GCN, HE-GCN GCN capture the hierarchical context of events, effectively separating features. This results a greater distance between feature clusters compared to Figure ((a) original features and Figure ((b) using only Euclidean representation learning. However, some challenging feature points remain difficult to distinguish. The addition of the DSI module facilitates information interactions between Euclidean and hyperbolic spaces, capturing more discriminative features to better differentiate these challenging feature points. As shown in Figure d), the DSI module further enhances feature differentiation by effectively combining information from both spaces.
Figure t-SNE visualization of the ablation module at feature-level.
Meanwhile, the frame-level, experiments conducted on two test videos as shown in Figure demonstrate that our method significantly improves the discriminative power for identifying violent frames compared to the baseline, which uses only GCN. Compared with the model w/o our core modules, both HE-GCN and DSI contribute to detecting violent frames.
Moreover, we provide more qualitative results, including the qualitative visualizations of VVD (Figure & and the qualitative visualizations of DSRL in the context of ambiguous violence (Figure 9
Qualitative visualizations.
Figure illustrates that DSRL can accurately distinguish between violent and normal events, demon- strating the effectiveness of DSRL. Additionally, compared with the baseline curves, our method shows better performance, further validating the effectiveness of the modules we designed.
16

---

wulo
w/o DSRL wa/
obsallation DoNLLoo croylo foy 100100000
(c) vtualiattoon DGR wi cradl II--ON Kultton 1000-6-511 00-47-30
05AL
DSRL w/o 051
CSRL w/ DSI
Ground Truth
Key frame of normality
Key frame of violence
Figure 7: Qualitative results of DSRL w/ or w/o our core modules (HE-GCN and DSI) for the test video from VioVenee dataset at frame-level.
Baseline
normality
Key frame violence
Figure 8: Frame-level scores and violence localization examples for the test video from XD- Violene dataset.
Qualitative Visualizations of DSRL in the context of ambiguous violence.
Figure shows that in the DD-Violence test set, DSRL accurately detects the correct categories of ambiguous violence, while using only Euclidean space or only hyperbolic space fails correctly detect these instances.
F Broader impacts
Potential positive societal impacts.
Our work can be more effective identifying incidents violence, albeit may be ambiguous. This contributes to higher level of safety in the community and the public's sense of security. Potential negative societal impacts. If our algorithm were to be widely applied in monitoring and law enforcement domains, it could
17

---

(x) Wern Morhwai 200
Welnehhcstin 201
31 17.22
Furlddaan
Hygeebolle
DSRL
Ground Trudh
Amb Frame
Figure 9: Qualitative Visualizations of DSRL in the context of ambiguous violence. The blue curves show violence scores predicted using only Euclidean representation, the yellow curve shows scores using only hyperbolic representation, the green curves show scores predicted by DSRL, and the pink area represents the ground-truth violent temporal location.
exacerbate societal surveillance and control, triggering concerns among the public regarding individual freedom and privacy rights.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the main claims and contributions the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please Sec
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
19

---

Justification:
We provide the full proof of Theorem in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced
statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Yes, the paper includes a detailed description of the experimental setup, methodology, and parameters used, ensuring that the main experimental results can be reproduced. Additionally, we provide comprehensive technical details in the appendix to support reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
20

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification:
We will open-source the code in the future.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //ipss.cc/pill wttpccppppppppppeeeeeeeeeeeeeeee httpa///guddeee/eeeeebiiissssss Coooeooomissssssseeeeeee ssiiiiiiccyyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: We provide detailed information on the experimental setup in the implementa- tion details section of the paper. See Sec. Cin the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Please see Sec. and Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for
example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
21

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer:[ Yes]
Justification: Please see Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please see Sec F
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
22

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [No]
Justification: The used datasets are publicly available, and widely used in existing VVD methods. Thus, we do not mention the license of the dataset.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
23

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's s creators.
13. New Assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We will open source the code after the paper is accepted.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [No]
Justification: The datasets used in this paper, XD-Violence and UCF-Crime are publicly available and widely used in research. These datasets were collected by their original creators and made accessible for research purposes.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [No]
Justification: Since we are using datasets that are already publicly available and have been extensively used in previous research, and given that the content does not involve sensitive personal information, this study did not undergo an independent IRB review.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
24

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
25


Reviews:
[{"review_id": "ji0B8qYTZD", "reviewer": "Reviewer_8atP", "summary": "The paper introduces a novel method called Dual-Space Representation Learning (DSRL) aimed at enhancing the detection of video violence, particularly in scenarios where the violence is weakly supervised and visually ambiguous. \n\nDSRL combines the strengths of both Euclidean and hyperbolic geometries to capture discriminative features for violence detection, leveraging the hierarchical structure modeling capability of hyperbolic spaces. \n\nTwo specialized modules are designed: the Hyperbolic Energy-constrained Graph Convolutional Network (HE-GCN) module and the Dual-Space Interaction (DSI) module, enhancing the understanding of event hierarchies and promoting dual-space cooperation.\n\nComprehensive experiments on the XD-Violence dataset demonstrate the effectiveness of DSRL, outperforming existing methods in both unimodal and multimodal settings.", "questions": "1. What are the specific reasons for the design choices in the Layer-Sensitive Hyperbolic Association Degree (LSHAD) with its multiple hyperparameters and threshold criteria? Was there any experimental or theoretical analysis conducted to support these design decisions?\n2. Considering the increased demand for computing resources and training time, are the benefits of these complex designs justified?\n3. What methods or heuristic approaches are used to determine the optimal values of these hyperparameters?  Has a systematic sensitivity analysis been conducted for the selection of hyperparameters (such as β, γ, α, etc.) in the DSRL model?\n4. Has the DSRL model been tested on other types of video content analysis tasks? If so, how did the model perform in these tasks? Are there any additional adjustments or optimizations required to adapt the model to different video content analysis applications?", "limitations": "The authors discussed their limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The method is innovative. DSRL designs a special information aggregation strategy. Through layer-sensitive hyperbolic association degree (LSHAD) and hyperbolic Dirichlet energy (HDE), it effectively captures the hierarchical context of events and can better model the problem of violent video detection.\n\n2. This method uses the property of hyperbolic space that can better distinguish visually similar events and is applied to a suitable scenario - fuzzy violent event recognition, which improves recognition accuracy.\n\n3. The performance of the method is outstanding. It has reached SOTA in quantitative analysis. In qualitative analysis, DSRL demonstrates its ability to distinguish between violent and normal events in different situations, including its performance in complex violent events.\n\n4. The proposed model can process multimodal inputs, integrate visual and audio information, and improve the ability to understand complex scenes.", "weaknesses": "Subjectivity: The Layer-Sensitive Hyperbolic Association Degree (LSHAD) proposed by the author contains multiple hyperparameters, and LSHAD is used to examine the threshold of the message graph. It is very important. There is a lack of explanation for why it is designed in this way.\n\nComplexity: DSRL combines Euclidean and hyperbolic geometric spaces, as well as cross-space interaction mechanisms, which may increase the complexity of the model, resulting in higher demands on computing resources and training time. \n\nParameter sensitivity: DSRL contains multiple hyper-parameters, such as β, γ, α, etc. The selection of these parameters may have a significant impact on model performance. This article lacks an analysis of parameter adjustment.\n\nApplication-specific limitations: DSRL is specifically designed for the task of video violence detection, which may mean that further adjustments or optimizations are required when applying it to other types of video content analysis tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the comments and suggestions to improve this work. We will address your concerns below.  \n   \n***Q1: Reasons for the design choices in the LSHAD with its multiple hyperparameters and threshold criteria*** \n     \nInspired by the Global-first principle [1] that humans always have cognition on global first and then focus on local, we propose a novel node selection strategy, which guarantees the model captures the broader global context first with a relaxed threshold at the beginning of message aggregation and then focuses on the local context with more strict thresholds. \nTo achieve this, we introduce the *LSHAD* construction rule, which calculates an *LSHAD* threshold based on the number of the current layer *k* and hyperbolic Dirichlet energy of the current layer. As the *k* increases and the hyperbolic Dirichlet energy decreases, the *LSHAD* threshold increases and is limited to [0,1] by the sigmoid function. If there is no $\\beta$ and $\\gamma$, the threshold in the first layer will be strict ($\\textgreater 0.5$), causing the overlook of some global context information. Therefore, to make our node selection threshold conform to the Global-first principle, we introduced the two hyperparameters in *LSHAD*, where $\\beta$ controls the influence of the number of current layer *k* and $\\gamma$ acts as a bias to fine-tune the threshold. Moreover, we conducted an ablation study to determine the optimal value of the two hyperparameters ($\\beta$,$\\gamma$), where $\\beta$ ranges among [0.2,0.4,0.6,0.8,1.0] and $\\gamma$ ranges among [1.0,1.2,1.4,1.6,1.8,2.0]. The results in the table below reveal that when $(\\gamma - \\beta)$ is 0.4, the performance is optimal, so we chose a pair (0.8, 1.2) from this set.\n| $\\beta \\setminus \\gamma $ | 1.0  | 1.2  | 1.4  | 1.6  | 1.8  | 2.0  |\n|--------------------------|------|------|------|------|------|------|\n| **0.2**                      | 85.22| 87.12| 86.32| 86.60 | 86.31| 86.86|\n| **0.4**                      | 87.52| 85.22| 87.12| 86.32| 86.60| 86.31|\n| **0.6**                      | 87.61| 87.52| 85.22| 87.12| 86.32| 86.60 |\n| **0.8**                      | 87.29| 87.61| 87.52| 85.22| 87.12| 86.32|\n| **1.0**                      | 86.29| 87.29| 87.61| 87.52| 85.22| 87.12|\n\n[1] Chen, L. (1982). Topological structure in visual perception. Science, 218, 699-700. \n  \n***Q2: Comparisons of computing resources and training time***   \n  \nOur designs, HE-GCN and DSI, effectively capture hierarchical contextual information of events and integrate two spaces of information, respectively.\nFollowing your suggestion, we trained three models on XD-Violence for 30 epochs each using a single NVIDIA RTX A6000 GPU: Baseline (GCN), Baseline+HE-GCN, and Baseline+HE-GCN+DSI (DSRL). The results in the table below show that the benefits of our designs are justified. Our DSRL improved AP by 3.57\\% compared to the Baseline, while the training time per epoch increased by only 41 seconds, and memory usage rose by just 4.1GB, both of which are within a reasonable range, making the performance gains well worth the resource consumption.   \n| Methods                    | Params  | Training Time per Epoch | Total Training Time | Video Memory Usage | AP (%) |\n|----------------------------|---------|:-------------------------:|:---------------------:|:--------------------:|:--------:|\n| Baseline (GCN)             | 0.7734M | 2 min                   | 60 min              | 4.24 GB            | 84.04  |\n| Baseline + HE-GCN          | 0.8975M | 2 min 19 s              | 69 min 39 s         | 7.03 GB            | 86.46  |\n| Baseline + HE-GCN + DSI (DSRL) | 0.9966M | 2 min 41 s              | 80 min 19 s         | 8.34 GB            | 87.61  |  \n\n***Q3: Hyperparameter sensitivity analysis***  \n  \nWe employed a grid search method to determine the optimal values of the hyperparameters in our model. We have conducted a sensitivity analysis on $\\alpha$ and $\\lambda$ in DSI, as shown in Line 470 in our submitted paper. \nWe further conducted a sensitivity analysis on $\\beta$ and $\\gamma$. The table referenced in **Q1** shows that our model is relatively robust to changes in the hyperparameters within certain ranges. \nBesides, the table also shows that when $(\\gamma - \\beta)$  is 0.4, the performance is optimal, so we chose the pair (0.8, 1.2) from this set.\nSpecifically, when $\\beta$=0.8, the variations in $\\gamma$ within the tested range do not significantly affect the model's performance. Similarly, when $\\gamma$=1.2, changes in $\\beta$  also have minimal impact on the model's effectiveness. This indicates that our model's performance is relatively stable under small perturbations of these hyperparameters, suggesting a degree of robustness in the parameter configuration.  \n  \n***Q4: Applicability to other video content analysis tasks***\n    \nCurrently, we primarily focus on addressing ambiguous violence in VVD by customising DSRL based on the dual-space idea. Although we have not yet tested its effectiveness on other tasks, our theoretical analysis suggests that this dual-space idea could be beneficial for various other video content analysis tasks. For instance, in video person re-identification (Video Re-ID), capturing both visual features and spatio-temporal hierarchical relationships is crucial. To adapt our model to Video Re-ID, we may consider the following adjustments:\n1) We may need to adjust the graph construction to fully incorporate information about human body parts and their hierarchical relationships.\n2) Video Re-ID is a type of fine-grained recognition that may require enhancing visual feature extraction in Euclidean space to strengthen node features in the graph.   \n\nIn the future, we will explore adapting the model to different video content analysis applications.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WfWXM9NMJv", "reviewer": "Reviewer_rxrS", "summary": "This paper proposes leveraging dual-space learning, encompassing both Euclidean and Hyperbolic spaces, to enhance discriminative capacity by capitalizing on the strengths inherent in hyperbolic learning. To overcome the limitations of the hard node strategy in the previous method, this work introduces DSRL (Dual-Space Representation Learning), which enhances node aggregation selection by utilizing layer-sensitive hyperbolic association degrees constrained by hyperbolic Dirichlet energy. A cross-space attention mechanism is then proposed to facilitate information interactions between Euclidean and hyperbolic space to capture better discriminative features.", "questions": "- Why utilize these two spaces (hyperbolic and Euclidean) for the representation learning? Have you explored different hyperbolic spaces such as Lorentz and Poincaré Ball?\n- How about the model's training stability?", "limitations": "Please refer to the weakness.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "### Novelty: \nHyperbolic space learning is a valuable direction in the field of video understanding because there are inherent hierarchical relationships under video series. The previous method (HyperVD) is a good attempt, but it simply transfers the XD-Violence baseline into hyperbolic space without solving the limitations in hyperbolic GCN. The authors introduce a dual-space approach to combine the strengths of both Euclidean and Hyperbolic spaces to maximize discrimination capability and use hyperbolic Dirichlet energy to address the over-smoothing issue underlining the previous hyperbolic networks.\n### Clarity: \nOverall, the main paper is easy to follow and organized well.\n### Experiments:\nThe main quantitative and qualitative experiments are adequate for the violence detection task, including ablation studies and t-SNE visualizations. The results demonstrate significant improvements compared to previous methods.", "weaknesses": "### Experiments:\nIt would have made the manuscript more convincing if the authors could provide inference visualizations of the ablation modules, which means to better illustrate the discriminative power of the method w/ or w/o your core modules.\n\n### Minor Typos:\n- Line 137:  $\\mathcal{L}^{\\/}$ should be $\\mathcal{L}^{n}$.\n- Eq (17) misses the right bracket for the $softmax$ function.\n- In Eq (9), it misses the symbol annotations for $\\mathbf{v}$. Did you use the hyperbolic transformation strategy proposed in \"Fully hyperbolic neural networks\", if so, the citation is missed here.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for acknowledging the strength of our method. We have carefully considered your constructive and insightful comments and here are the answers to your concerns.  \n\n***Q1: It would have made the manuscript more convincing if the authors could provide inference visualizations of the ablation modules, which means to better illustrate the discriminative power of the method w/ or w/o your core modules.***\n  \nThank you for your positive comments and valuable suggestions. We have supplemented inference visualization results for the ablation module, which will be added in the final version. We conducted corresponding visualisation experiments to explore the discriminative power of the model w/ or w/o our core modules, HE-GCN and DSI. **Specific visualisations and analysis are shown in Figures 1 and 2 of the uploaded PDF file.** Two dimensions (the feature-level and the frame-leve) are analyzed in the two figures.     \n**At the feature-level,** the results are shown in Figure 1 of the uploaded PDF. Compared to GCN, HE-GCN can capture the hierarchical context of events, effectively separating features. This results in a greater distance between feature clusters compared to Figure 1(a) original features and Figure 1(b) using only Euclidean representation learning. However, some challenging feature points remain difficult to distinguish. The addition of the DSI module facilitates information interactions between Euclidean and hyperbolic spaces, capturing more discriminative features to better differentiate these challenging feature points. As shown in Figure 1(d), the DSI module further enhances feature differentiation by effectively combining information from both spaces.   \nMeanwhile, **at the frame-level**, experiments conducted on two test videos as shown in Figure 2 of the uploaded PDF demonstrate that our method significantly improves the discriminative power for identifying violent frames compared to the baseline, which uses only GCN. Compared with the model w/o our core modules, both HE-GCN and DSI contribute to detecting violent frames.  \nFollowing your suggestions, we believe that we can more comprehensively demonstrate the effectiveness of our method and make our manuscript more convincing.\n\n***Q2: Minor Typos:***  \n  \nThank you for spotting these minor typos. We will modify them accordingly in the final version. In Eq(9), $\\textbf{v} \\in \\mathbb{R}^{n+1}$ denotes a velocity (ratio to the speed of light) in the Lorentz transformations. And we used the hyperbolic transformation from this paper, which we will cite in the final version. \n\n***Q3: Why utilize these two spaces (hyperbolic and Euclidean) for the representation learning? Have you explored different hyperbolic spaces such as Lorentz and Poincaré Ball?***  \n  \nSince we focus on addressing ambiguous violence in the VVD task, it is essential to consider both visual features and the hierarchical contextual information of the event. Therefore, we employ these two spaces for representation learning for two main reasons:\n1) Euclidean space is widely used in many domains and is effective at capturing visual features, such as salient motion and shape changes in videos. However, it often overlooks the relationships between events.   \n2) Hyperbolic space, characterized by exponentially increasing metric distances, naturally reflects the hierarchical structure of data. It enhances the hierarchical relationships of events but tends to weaken the expression of visual features.  \n\nBy combining these two spaces for representation learning, we can leverage their respective strengths to improve the discriminative of feature representations, ultimately enhancing the performance of VVD.  \n\nIn the preliminary practice, we implement our method with the Poincaré ball model.  However, we encountered situations where the loss became NaN, which is caused by the numerical instability of the Poincaré ball model. Subsequently, we switched to the Lorentz model because it guarantees numerical stability and computational simplicity in its exponential and logarithmic maps and distance functions. Due to the stable training and improved performance, we finally selected the Lorentz model during hyperbolic representation learning. \n\n***Q4: How about the model's training stability?***  \n  \nWe evaluate the training stability of our model by analyzing the trends in training loss and average precision (AP). **The results presented in Figure 3 of the uploaded PDF file** exhibit a stable training process.      \n\n**Training loss curve over time.** The training loss starts at approximately 0.61 and gradually decreases over time, stabilizing around 0.17 towards the end of the training. This consistent decrease and eventual stabilization indicate effective learning and convergence throughout the training process.     \n**Average precision curve over time.** The AP shows some fluctuations during the early epochs, which is normal as the model adjusts its weights and parameters. Despite these fluctuations, the overall trend is upward, indicating that the model's performance is improving. Subsequently, the AP values show less variation and remain at a high level.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bIvmkazaDv", "reviewer": "Reviewer_5VkS", "summary": "This paper presents a novel approach called Dual-Space Representation Learning (DSRL) for weakly supervised Video Violence Detection (VVD). Traditional VVD methods rely heavily on Euclidean space representation, which often fails to distinguish between visually similar events. The proposed DSRL method leverages both Euclidean and hyperbolic geometries to enhance the discriminative capacity of features. It introduces two key modules: the Hyperbolic Energy-constrained Graph Convolutional Network (HE-GCN) and the Dual-Space Interaction (DSI) module, to facilitate better information interactions and improve violence detection accuracy. The method achieves state-of-the-art performance on the XD-Violence dataset in both unimodal and multimodal settings. Additionally, the method shows good effectiveness on the UCF-Crime dataset, further proving its strong generalization ability across datasets.", "questions": "Same as weakness.", "limitations": "N/A", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The proposed method is totally interesting and novel. It provides an effective solution to the ambiguous anomalies problem by progressively learning event context in hyperbolic spaces.\n\n2.The proposed method achieves good performance, especially on XD-Violence dataset in both unimodal and multimodal settings. Visualization experiments also highlight its effectiveness in solving the ambiguous anomalies problem.\n\n3.The HE-GCN and DSI modules are well-motivated, and the authors have reported sufficient ablation study results to prove the effectiveness of each module. I think it is much helpful.", "weaknesses": "1.For the HE-GCN module, I am curious about the relationship between HDE and LSHAD. The authors should explain the relationship between HDE and LSHAD in detail?\n\n2.Figure 2 should be revised to clarify that the method is applicable to both unimodal and multimodal inputs, not just multimodal. Currently, it may mislead readers.\n\n3.The Preliminaries section is well-detailed and informative, but it may be complex and difficult for a broader audience to understand.\n\n4.There are some minor writing issues: Lines 439 and 479, “Figure” and “Fig” should be standardized; Line 147, there is a symbol display error.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "***Q1: For the HE-GCN module, I am curious about the relationship between HDE and LSHAD. The authors should explain the relationship between HDE and LSHAD in detail?***\n   \nThank you for the valuable comments. We will elaborate on the relationship between *HDE* and *LSHAD* in detail. \nIn fact, *HDE* was designed to serve *LSHAD*. To enhance the model's ability to capture contextual information in hyperbolic space, we propose *LSHAD* as a threshold in the node selection for message aggregation, which ensures that the model first captures the broader global context with a relaxed threshold at the beginning of message aggregation, then gradually shifts focus to the local context with stricter thresholds.\nWhen there is a significant difference in features between nodes, we need to adopt a more relaxed node selection strategy to choose as many nodes as possible. However, as the number of layers increases and the features between nodes become more similar, we need to adopt a stricter selection strategy to choose the more important nodes. \nTherefore, we make the *LSHAD* relate to the degree of message aggregation of the current layer, which is measured by *HDE*. *HDE* essentially captures how effectively the nodes' information is aggregated and how similar their features become through the message-passing process.\nThe synergy between *HDE* and *LSHAD* ensures that HE-GCN module effectively captures hierarchical relationships and aggregates information, enhancing the discriminative power of DSRL. By employing these strategies, our DSRL leverages the strengths of both Euclidean and hyperbolic geometries to improve the performance of VVD, particularly in distinguishing ambiguous violent events.\n\n***Q2: Figure 2 should be revised to clarify that the method is applicable to both unimodal and multimodal inputs, not just multimodal. Currently, it may mislead readers.***  \n\nWe thank the reviewer for the valuable suggestions. We will modify Figure 2 to clarify that our method is applicable to both unimodal and multimodal inputs in the final version.\n\n***Q3: The Preliminaries section is well-detailed and informative, but it may be complex and difficult for a broader audience to understand.***\n  \nThank you very much for your valuable feedback. We understand that the Preliminaries section may be complex for a broader audience, and we are committed to making it more accessible. In the final version of the paper, we will add further explanations for some of the equations, such as providing additional descriptions for Eq. 4 and Eq. 5.   \nWe have revised the explanation regarding the connections between hyperbolic space and tangent space. The original sentence:\n\"The connections between hyperbolic space and tangent space are established by the exponential map $\\exp _{\\mathbf{x}}^{K}(\\cdot)$ and logarithmic map $\\log _{\\mathbf{x}}^{K}(\\cdot)$ are given as follows:\" has been modified for clarity.\n\nThe revised explanation is: \"The mapping between hyperbolic spaces and tangent spaces can be done by exponential map and logarithmic map. The exponential map is a map from a subset of a tangent space of $\\mathbb{L} _ {K}^{n}$ (i.e., $\\mathcal{T} _ {\\mathbf{x}} \\mathbb{L} _ {K}^{n}$) to  $\\mathbb{L} _ {K}^{n}$ itself. The logarithmic map is the reverse map that maps back to the tangent space. For points $\\textbf{x} ,\\textbf{y}  \\in \\mathbb{L} _ {K}^{n} $,  $\\textbf{v}\\in \\mathcal{T} _ {\\mathbf{x}} \\mathbb{L} _ {K}^{n}$, such that $\\textbf{v}\\ne \\textbf{0} $ and $\\textbf{x}\\ne \\textbf{y} $, the exponential map $\\exp _{\\mathbf{x}}^{K}(\\cdot)$ and logarithmic map $ \\log _{\\mathbf{x}}^{K}(\\cdot) $ are given as follows:''.   \n\n***Q4: There are some minor writing issues: Lines 439 and 479, “Figure” and “Fig” should be standardized; Line 147, there is a symbol display error.***\n  \nThank you for your valuable comments. We will address these minor writing issues in the final version by changing \"Fig\" to \"Figure\" and correcting the symbol errors. Furthermore, we have carefully reviewed the paper to ensure that there are no writing errors.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CNGhJeeMlw", "reviewer": "Reviewer_8SDB", "summary": "This paper presents a pioneering approach called Dual-Space Representation Learning (DSRL) for the task of weakly supervised video violence detection.  The overall framework contains the Hyperbolic Energy-constrained Graph Convolutional Network (HE-GCN) for capturing event hierarchies and the Dual-Space Interaction (DSI) module to facilitate cross-space feature integration. Experiments on XD-Violence and UCF-Crime datasets show the advantage of the proposed method.", "questions": "- **Qualitative Visualizations**: Does the 'Hyperbolic' in Figure 5 represent the result of HyperVD? If not, I wonder whether HyperVD can handle ambiguous violence.", "limitations": "Please see #Weaknesses and #Questions.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Motivation**: The authors propose to combine Euclidean and hyperbolic geometries to handle the challenging scenarios of ambiguous violence, which is a problem-centric motivation.\n- **Written**: The quality of this paper's presentation is good and the whole paper is well-organized.\n- **Technical Correctness**: The proposed method is technically sound and has been evaluated on two datasets.", "weaknesses": "- **Novelty**: Some modules in this paper lack innovation, such as the cross-graph attention mechanism is commonly used in previous works.\n- **Model Complexity**: This paper does not discuss the computational complexity or parameters of the DSRL model. For practical applications, especially in real-time surveillance scenarios, it's important to ensure that the model can operate efficiently with minimal latency.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive evaluation of the paper. We have carefully considered your constructive and insightful comments and here are the answers to your concerns.   \n\n***Q1: Novelty: Some modules in this paper lack innovation, such as the cross-graph attention mechanism is commonly used in previous works.***   \n   \nThank you for your feedback regarding the novelty of our work. Here we would like to emphasize the innovative aspects of the introduced Cross-Space Attention (CSA) mechanism and Hyperbolic Energy-constrained Graph Convolutional Network Module (HE-GCN).  \nCompared with the commonly used cross-graph attention (CGA) mechanism,   \n**i) the role of our CSA is different.** CSA aims to break the information cocoon of different spaces and realize the interaction of different geometric structure information across spaces while CGA focuses on the representation learning of the same geometric structure information in a single space.   \n\n**ii) The interaction strategy of CSA is different.** On the one hand, since the Lorentzian metric preserves true relationships by computing the nonlinear distance between nodes, while the cosine similarity may show fake relationships, especially for ambiguous violence with similar visual cues, we adopt the Lorentzian metric instead of the cosine similarity that is commonly used in CGA to obtain attention weights for better interaction in spaces with different geometric structures. The corresponding comparison results reported in the submitted manuscript (lines 270-274) also prove the effectiveness of the Lorentzian metric. On the other hand, we use a step-by-step interaction model. Unlike the commonly used one-step interaction, the result of one-step interaction, which is dominated by the information of the current space, is interacted again with the original information in another space to achieve full cross-space information interaction.  \n\n**iii) The construction of the graph for interaction is different.** Unlike the traditional strategy of node selection for message aggregation, the graph in CSA is constructed through a more effective dynamic node selection strategy where the node selection threshold during message aggregation is layer-sensitive and affected by the hyperbolic Dirichlet energy of the current layer. \n\nIt is worth emphasizing that HE-GCN is also one of our main innovations. Instead of adopting the hard node selection strategy in HGCN, HE-GCN selects nodes for message aggregation by our introduced layer-sensitive hyperbolic association degrees, which are dynamic thresholds determined by the message aggregation degree at each layer. To better align with the characteristics of hyperbolic spaces, we introduce the hyperbolic Dirichlet energy to quantify the extent of message aggregation. Benefiting from the dynamic threshold, the layer-by-layer focused message passing strategy adopted by HE-GCN not only ensures the efficiency of information excavation but also improves the model's comprehensive understanding of the events, thus enhancing the model's ability to discriminate ambiguous violent events.   \n\n***Q2: Model Complexity: This paper does not discuss the computational complexity or parameters of the DSRL model. For practical applications, especially in real-time surveillance scenarios, it's important to ensure that the model can operate efficiently with minimal latency.*** \n    \nThank you for the valuable feedback regarding the computational complexity and parameters of the DSRL model. We agree that understanding the model's complexity is especially important for real-time applications. Our model indeed meets the requirements for real-time processing, as analyzed below:   \nOur experiments were conducted on a single NVIDIA RTX A6000 GPU.  \nFor **Video Input**, the model achieves 83.87 FPS while handling only video data. The model's parameters are 13.4 MB (I3D parameters at 12.49 MB and DSRL parameters at 0.91MB), which keeps the overall parameter size manageable and enables quick response times.   \nFor **Video + Audio Input**, the model maintains a high processing speed of 56.86 FPS, even with the additional computational load of audio processing. The model's parameters are 85.54 MB (I3D parameters at 12.49 MB, VGGish parameters at 72.14 MB and DSRL parameters at 0.91MB ). \n\nThese results demonstrate that our model exhibits excellent real-time performance with multimodal inputs, making it suitable for latency-sensitive real-world applications.\n\n***Q3: Does the \"Hyperbolic\" in Figure 5 represent the result of HyperVD?  If not, I wonder whether HyperVD can handle ambiguous violence.***   \n\nYes, the \"Hyperbolic\" in Figure 5 indeed represents the result of HyperVD. \nOur experiments indicate that HyperVD is not sufficiently effective in handling ambiguous violence due to two main limitations. First, HyperVD relies solely on hyperbolic representations, which weakens its ability to capture essential visual features. Second, it inadequately learns the hierarchical relationships of complex violent events, which further contributes to its suboptimal performance in dealing with ambiguous violence. We will clarify in the final version of the paper that \"Hyperbolic\" refers to the results of HyperVD.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all the reviewers for providing constructive feedback that helped us improve the paper. We are glad the reviewers find that

 **Our method is interesting and novel**

* "The method is innovative." ---8atP
* "The proposed method is totally interesting and novel." ---5VkS

* "This paper presents a pioneering approach." ---8SDB
 
**Our presentation is clear and well-motivated**

* "Overall, the main paper is easy to follow and organized well." ---rxrS

* "The HE-GCN and DSI modules are well-motivated." ---5VkS

* "The quality of this paper's presentation is good and the whole paper is well-organized"---8SDB

**Our performance is outstanding**

* "The performance of the method is outstanding. " ---8atP

* "The results demonstrate significant improvements compared to previous methods. " ---rxrS

*  "The proposed method achieves good performance " ---5VkS

*  "Experiments on XD-Violence and UCF-Crime datasets show the advantage of the proposed method. " ---8SDB

**Main contributions** 

1. We present DSRL, the first method to integrate Euclidean and hyperbolic geometries for VVD, which significantly improves the discrimination of ambiguous violence and achieves state-of-the-art performance on the XD-Violence dataset in both unimodal and multimodal settings.

2. We design the HE-GCN module with a novel message aggregation strategy to better capture the hierarchical context of events, where the node selection threshold is dynamic, not fixed, and determined by layer-sensitive hyperbolic association degrees based on hyperbolic Dirichlet energy.

3. We design the DSI module to break the information cocoon for better dual-space cooperation, combining visual discrimination from Euclidean geometry and event hierarchical discrimination from hyperbolic geometry, utilizing cross-space attention to facilitate information interactions.

Once again, we express our sincere appreciation for your valuable contributions to the review process. Your expertise and guidance have been invaluable in improving the quality of our work. We remain committed to continuous improvement and eagerly await your final decision. Please see the attached PDF for a one-page PDF with added experimental results.

Meta-review: The paper proposes a method for violence detection from videos. The reviewers found the method interesting, novel and effective. The evaluation shows the effectiveness of the proposed method. The authors rebuttal further clarified questions from the reviewers. On the other hand, it is very important to acknowledge and discuss the issue of potential model bias. For example, the learned model may work better just by taking advantage of false association between gender/race and violence. This has been shown in many related works, such as racial bias in hate speech detection. It is very strongly recommended that the authors discuss this issue in details and also perform additional analysis showing the model (or the improvement) is not biased against protected attributes.
Decision: Accept (poster)