Beyond Euclidean: Dual-Space Representation Learning for Weakly Supervised Video Violence Detection

Authors: jiaxu  leng
zhanjie  wu
mingpi  tan
yiran  liu
ji  gan
haosheng  chen
xinbo  gao
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 08:11:45

Abstract: While numerous Video Violence Detection (VVD) methods have focused on representation learning in Euclidean space, they struggle to learn sufficiently discriminative features, leading to weaknesses in recognizing normal events that are visually similar to violent events (i.e., ambiguous violence). In contrast, hyperbolic representation learning, renowned for its ability to model hierarchical and complex relationships between events, has the potential to amplify the discrimination between visually similar events. Inspired by these, we develop a novel Dual-Space Representation Learning (DSRL) method for weakly supervised VVD to utilize the strength of both Euclidean and hyperbolic geometries, capturing the visual features of events while also exploring the intrinsic relations between events, thereby enhancing the discriminative capacity of the features. DSRL employs a novel information aggregation strategy to progressively learn event context in hyperbolic spaces, which selects aggregation nodes through layer-sensitive hyperbolic association degrees constrained by hyperbolic Dirichlet energy. Furthermore, DSRL attempts to break the cyber-balkanization of different spaces, utilizing cross-space attention to facilitate information interactions between Euclidean and hyperbolic space to capture better discriminative features for final violence detection.  Comprehensive experiments demonstrate the effectiveness of our proposed DSRL.

Main paper: 
 introduction: 
 Compared with manually checking out violent events in surveillance videos which is time-consuming and laborious, Video Violence Detection (VVD), which focuses on identifying violent events in videos and provides automatic and instantaneous responses, has gained significant research attention due to its potential applications. However, it is expensive to annotate each frame in a video so that we can train a VVD model with supervised learning. To address this, current methods often utilize weakly supervised settings to formulate the problem as a multiple-instance learning (MIL) [19] task. These methods treat a video as a bag of instances (i.e., snippets or segments), and predict their labels based on the video-level annotations.According to the modality type of the input data, existing weakly supervised VVD methods can be roughly divided into two categories, unimodal with only vision input and multimodal with vision and audio input. The unimodal methods [29, 36, 8, 10, 20] learn the different distributions of normal and violent events through video-level labels, focus on finding valuable visual cues that are distinct from non-violent events and use them to detect violent events, i.e., fighting. However, relying on visual cues The hierarchy of event categoriesThe hierarchy of event development to identify violent events is sometimes unreliable, especially when facing visually ambiguous events, like normal physical collisions and fighting behavior in hockey games. To alleviate this problem, Wu et al [34] released a large multimodal violence dataset, named XD-Violence, and accelerated a series of multimodal VVD methods [34, 35, 22, 37, 23] . Multimodal VVD methods incorporate not only visual cues but also complementary audio information for improving the discrimination of violent events. Previous methods have employed Euclidean space representation learning and achieved good results in many other computer vision tasks [27, 28, 15, 32] . Despite advancements in audio-visual violent video detection (VVD) methods, their performance remains unsatisfactory in recognizing normal events that are visually similar to violent events (i.e., ambiguous violence) due to the limitations of Euclidean space. While visual features are fully extracted in Euclidean space, these methods fail to adequately capture and utilize the intrinsic relations between events.To fully understand an event, on the one hand, we need to explore the hierarchy of event categories, on the other hand, we need to sort through the events, including the trend before the event, the action during the event and the behaviour after the event, which reflects the hierarchy of event development, shown in Figure 1 (a). An ambiguous violent event is confusing at the current category level or happening moment, but may easily be detected if we can grasp the hierarchical relations. Fortunately, hyperbolic representation learning, characterized by exponentially increasing the metric distances and naturally reflects the hierarchical structure of data, has gained attention and shown promising performance in computer vision tasks, like semantic segmentation [1] , medical image recognition [38] , action recognition [24, 17] , anomaly recognition [12] . At present, only one method, HyperVD [25] , makes a preliminary attempt on the VVD task via hyperbolic representation learning. HyperVD introduces the Hyperbolic Graph Convolutional Networks (HGCN) [3] , an extended version of Euclidean graph convolution for representation learning in hyperbolic space, to learn discriminative representations. However, HGCN employs a hard node selection strategy during the message passing, where the nodes whose correlation is higher than the threshold (a manual parameter) are selected for message aggregation and otherwise discarded, which leads to insufficient hierarchical relation learning. In addition, existing VVD methods deploy feature embedding either in Euclidean or hyperbolic spaces. Representation learning in a single space is like picking the sesame and losing the watermelon, where the feature embedding is insufficient to guarantee the performance of VVD. On the one hand, hyperbolic representation learning strengthens the hierarchical relation of events but weakens the expression of visual features, on the other hand, Euclidean representations emphasize visual features but ignore relations between events. Therefore, leveraging the advantages of both spaces is essential for improving the performance of VVD methods, shown in Figure 1 (b).In this paper, we propose a novel Dual-Space Representation Learning (DSRL) method for weakly supervised VVD under the multimodal input setting. Specifically, we designed two customized modules, the Hyperbolic Energy-constrained Graph Convolutional Network module (HE-GCN) and the Dual-Space Interaction module (DSI). Instead of adopting the hard node selection strategy in HGCN, HE-GCN selects nodes for message aggregation by our introduced layer-sensitive hyperbolic association degrees, which are dynamic thresholds determined by the message aggregation degree at each layer. To better align with the characteristics of hyperbolic spaces, we introduce the hyperbolic Dirichlet energy to quantify the extent of message aggregation. Benefiting from the dynamic threshold, the layer-by-layer focused message passing strategy adopted by HE-GCN not only ensures the efficiency of information excavation but also improves the model's comprehensive understanding of the events, thus enhancing the model's ability to discriminate ambiguous violent events. Although hyperbolic representation learning enhances the understanding of hierarchical relations of events, the role of visual representations in violence detection cannot be discarded. However, fusing representation in different spaces remains a challenge, to break the information cocoon, DSI utilises cross-space attention to facilitate information interactions between Euclidean and hyperbolic space to capture better discriminative features, where Euclidean representations have effectiveness on the significant motion and shape changes in the video, while hyperbolic representations accelerate the comprehension of hierarchical relations between events, working together to improve the performance of violence detection in videos.To the best of our knowledge, our DSRL is the first method to integrate Euclidean and hyperbolic geometries for VVD, significantly improving discrimination of ambiguous violence and achieving state-of-the-art performance on the XD-Violence dataset in both unimodal and multimodal settings.(2) To better capture the hierarchical context of events, we design the HE-GCN module with a novel message aggregation strategy, where the node selection threshold is dynamic not fixed and determined by layer-sensitive hyperbolic association degrees based on hyperbolic Dirichlet energy.(3) To break the information cocoon for better dual-space cooperation, visual discrimination from Euclidean and event hierarchical discrimination from hyperbolic, we propose the DSI module, which utilizes cross-space attention to facilitate information interactions.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The abstract and introduction clearly state the main claims and contributions of the paper. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
related work: 
 Weakly Supervised Video Violence Detection. Weakly supervised VVD requires identifying violent snippets under video-level labels, where the MIL [19] framework is widely used for denoising irrelevant information. Recently, progress has been made in weakly supervised VVD, with approaches categorized into two main categories: vision-based and audio-visual-based methods. Employing exclusively visual cues, vision-based VVD endeavours to discern the occurrence of violent events within videos. Most existing works [7, 26, 30, 33] consider VVD as solely a visual task, and CNNbased networks are utilized to encode visual features. However, these approaches overlook the interaction between different modalities and the relevant audio information, which could negatively impact the accuracy of violence detection. To alleviate this problem, Wu et al [34] released a large multimodal violence dataset, named XD-Violence, and accelerated a series of multimodal VVD methods [34, 35, 22, 37, 23, 18, 25] . In contrast to unimodal methods, they incorporate not only visual cues but also complementary audio information for improving the discrimination of ambiguous violent events. Subsequently, many studies [22, 23] have focused on the integration of visual and audio information. There is also work [37] focused on solving the modality asynchrony problem. Despite the progress of weakly supervised multimodal VVD methods, all the above methods carry out representation learning in Euclidean Spaces, making it difficult to effectively handle ambiguous violence.Hyperbolic Representation Learning. Hyperbolic representation learning, characterized by exponentially increasing the metric distances and naturally reflects the hierarchical structure of data, has gained attention and shown promising performance in computer vision tasks, like semantic segmentation [1] , visual representation learning [9] , medical image recognition [38] , action recognition [24, 17] , anomaly recognition [12] . More Recently, HyperVD [25] has made an initial attempt at the VVD task using hyperbolic representation learning. HyperVD incorporates Hyperbolic Graph Convolutional Networks (HGCN) [3] to acquire discriminative representations. However, while hyperbolic representation learning enhances the hierarchical relationships of events, it diminishes the representation of visual features. Therefore, we propose a novel Dual-Space Representation Learning to joint the strengths of Euclidean and hyperbolic space. Meanwhile, we design HE-GCN to gradually shift its focus from capturing global contextual information to concentrating on crucial detailed features, progressively capturing the hierarchical context of events.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Please see Sec.6. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
preliminaries: 
 Problem Definition. Given an video sequence S = {S t } T t=1 with T non-overlapping segments. For a video segment, the weakly supervised VVD requires distinguishing whether it contains violent events via an events relevance label y t ∈ {0, 1}, where y t = 1 means in the current segment includes violent cues. Hyperbolic Geometry. A Riemannian manifold (M, g) of dimension n is a real and smooth manifold equipped with an inner product on tangent space g x : T x M × T x M → R at each point x ∈ M, where the tangent space T x M is a n-dimensional vector space and can be seen as a first-order local approximation of M around point x. In particular, hyperbolic space (D n c , g c ), a constant negative curvature Riemannian manifold, is defined by the manifold D n c = {x ∈ R n : c ∥x∥ < 1} equipped with the following Riemannian metric:g c x = λ 2x g E , where λ x := 2 1-c∥x∥ 2 and g E = I n is the Euclidean metric tensor. Considering the numerical stability and calculation simplicity of its exponential and logarithmic maps and distance functions, we select the Lorentz model [21] as the framework cornerstone. Lorentz Model. Formally, an n-dimensional Lorentz model is the Riemannian manifoldL n K = L n , g K x . K is the constant negative curvature. g K x = diag (-1, 1, • • •, 1)is the Riemannian metric tensor. We denote L n as the n-dimensional hyperboloid manifold with constant negative curvature K:EQUATIONLet x, y ∈ R n+1 , then the Lorentzian scalar product is defined as:EQUATIONwhere L n is the upper sheet of hyperboloid in an (n+1)-dimensional Minkowski space with the origin -1/K, 0, • • •, 0 . For simplicity, we denote point x in the Lorentz model as x ∈ L n K . Tangent Space. Given the tangent space at x is defined as an n-dimensional vector space approximating L n K around x,EQUATIONNote that T x L n K is a Euclidean subspace of R n+1 . Particularly, we denote the tangent space at the origin as T 0 L n K . Logarithmic and Exponential Maps. The mapping between hyperbolic spaces and tangent spaces can be done by exponential map and logarithmic map. The exponential map is a map from a subset of a tangent space of L n K (i.e., T x L n K ) to L n K itself. The logarithmic map is the reverse map that maps back to the tangent space. For points x, y ∈ L n K , v ∈ T x L n K , such that v ̸ = 0 and x ̸ = y, the exponential map exp Kx (•) and logarithmic map log K x (•) are given as follows:EQUATIONwhere∥v∥ L = ⟨v, v⟩ L denotes Lorentzian norm of v and d K L (•, •) denotes the Lorentzian intrinsic distance function between two points x, y ∈ L n K , which is given as:EQUATIONQuestion: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [No] Justification: We will open-source the code in the future.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
methodology: 
 In this paper, we propose the Dual-Space Representation Learning (DSRL) method to improve the discrimination of ambiguous violence. Within DSRL, we first design the Hyperbolic Energyconstrained Graph Convolutional Network Module (HE-GCN) to better capture the hierarchical context of events (Sec. 4.1 ). Then, the Dual-Space Interaction Module (DSI) is introduced to break the information cocoon for better dual-space cooperation (Sec. 4.2 ). An illustration of the main components of DSRL is provided in Figure 2 . Association degreeEQUATION1 ) ( ) 1 ) ( 1 ( ) ), ( (      x E k sigmoid k x E f F K H K H   EV EA)) , ( ( ) , (j i K L j i x x d exp x x Ls   k j i j i j i LSHAD A A A   , , ,, ' We can reasonably regard 0, x E i as a node on the tangent space at the origin o. HE-GCN uses the exponential map defined in Eq. 4 to generate hyperbolic node representations on the Lorentz model:EQUATIONEQUATIONHyperbolic Feature Transformation. According to [5] , we reformalize the lorentz linear layer tolearn a matrix M = v ⊤ W , v ∈ R n+1 , W ∈ R m×(n+1) satisfying ∀x ∈ L n , f x (M)x ∈ L m, wheref x : R (m+1)×(n+1) → R (m+1)×(n+1) should be a function that maps any matrix to a suitable one for the hyperbolic linear layer. Specifically, ∀x ∈ L n K , M ∈ R (m+1)×(n+1) , f x (M) is given asEQUATION)Theorem 1 ∀x ∈ L n , M ∈ R (m+1)×(n+1) , we have f x (M)x ∈ L m K .For simplicity, we use a morel general formula * of hyperbolic linear layer for feature transformation based on f x v ⊤ W x with activation, dropout, bias and normalization,EQUATIONwhere x ∈ L n K , W ∈ R m×(n+1) , v ∈ R n+1EQUATIONwhere d i/j denotes the node degree of node i/j. The distance d K L (x, y) between two points x, y ∈ L is the geodesic. Given that each node is connected to every other node in videos, resulting in a node degree d i of n -1 (where n is the total number of nodes), the formula for hyperbolic Dirichlet energy can be simplified.EQUATION)HDE is used to measure the similarity between node features in order to gauge the degree of information aggregation among features. It is evident that as hyperbolic message aggregation progresses, the similarity between features gradually decreases. Hyperbolic message aggregation reduces HDE. It can be expressed as:E K H x (l+1) ≤ E K H x (l), where l is the layer number. Layer-Sensitive Hyperbolic Association Degree. Based on HDE, we design Layer-Sensitive Hyperbolic Association Degree (LSHAD) to guide the node selection strategy for our construction of graphs for message aggregation. It is defined asEQUATIONwhere f (•) is a function related to k and E K H (x), k is the current layer number. β and γ are hyperparameters. Lorentzian similarity. Based on the Lorentzian distance, the Lorentzian similarity to measure the feature semantic similarity between nodes is given byEQUATIONwhered K L (•, •) is the Lorentzian intrinsic distance function.We define the initial adjacent matrix A L ∈ R T ×T via lorentz similarity:EQUATIONLSHAD Construct rules. With LSHAD, we propose message graph construction rules, called LSHAD construct rules. It is defined asEQUATIONwhere A i,j means the lorentz similarity between nodes i and j in the graph G. Formally, it enforces the elements of the adjacency matrix A that are less than LSHAD to be zeros. Finally, we can dynamically construct message graphs at each layer via LSHAD construct rules to obtain better contextual information. Following LSHAD construction rules, we can dynamically construct message graphs G ′ , which we then use to perform message aggregation operations to obtain better contextual information. Hyperbolic Message Aggregation. We use graph G ′ to perform message aggregation, and the message aggregation can be defined as:EQUATION)where m is the number of nodes. y i is the node features.Although hyperbolic representation learning enhances understanding of event hierarchies, visual representations remain crucial in violence detection. Fusing representations from different spaces is challenging; thus, DSI employs cross-space attention to facilitate interactions between Euclidean and hyperbolic spaces.Cross-Space Attention Mechanism. Cross-Space Attention Mechanism utilizes the Lorentzian metric to calculate attention scores between nodes from different spaces, accurately measuring semantic similarity and better preserving their true relationships by computing the nonlinear distance between them. We denote the features in Euclidean space as V E and the features in hyperbolic space as V H . CSA E→H models the between-graph interaction and guides the transfer of inter-graph message from V E to V H . First, we use linear layer to transform V H to the key graph V k and the value graph V v ,and V E to the query graph V q . Then, we use lorentzian metric to calculate the attention map A E→H as follows:EQUATIONwhere Ls(•) is Lorentzian similarity defined in Eq. 13 and λ is the threshold value to eliminate weak relations and strengthen correlations of more similar pairs. The representation from E to H can be formulated as follows:EQUATIONThe interaction process in DSI can be represented as follows:EQUATIONwhere V ′ E represents the features obtained by enhancing Euclidean space features using hyperbolic space features, and α controls the contribution of the enhanced features and V E . M axP ool is the max pooling operation and ⊕ represents the concatenation operation. Learning Objective. We use binary cross-entropy as our classification loss. Its calculation formula is:EQUATIONwhere y i is true label, ŷi is the predicted label, N is the batch size. 
experiments setup: 
 Datasets. Under the multimodal input setting, we follow [34, 37, 25] to conduct experiments on XD-Violence, which is the only and extremely challenging VVD dataset with multimodal information. Under the unimodal input setting, both the XD-Violence [34] and UCF-Crime [30] datasets are used to evaluate our method. More details of the two dataset settings are provided in the Appendix. Evaluation Metrics. To quantitatively evaluate the performance, we follow standard pratice [34, 37, 18, 6] . For XD-Violence, we utilize the frame-level average precision (AP) as the evaluation metric.For UCF-Crime, we adopt the area under the curve of the frame-level receiver operating characteristic (AUC) to evaluate performance.In Table 1 , with multimodal input, our DSRL demonstrates superior performance on XD-Violence, surpassing the best method that uses Euclidean space representation by 4.21% and that uses only hyperbolic space representation by 1.94%. These results highlight the effectiveness of DSRL in learning discriminative features and prove that our dual-space representation learning integrates the benefits of Euclidean and hyperbolic space well. From Table 1 , it also can be found that DSRL achieves SOTA performance under unimodal input settings on XD-Violence, outperforming existing state-of-the-art methods. Furthermore, to analyse the generalization of our method, we also report the performance on the UCF-Crime dataset, achieving an accuracy of 86.38%, comparable to current state-of-the-art methods.We conduct ablation studies on various design choices of our DSRL to demonstrate their contributions to the final results in Table 2 . 1) Component-wise ablations. To study the impact of each component in the DSRL, including HE-GCN and DSI, we start with a baseline model that only applies GCN and progressively adds each component. Table 2 shows that the baseline yields inferior performance due to learning representations only in Euclidean space (1 st row). Then, HE-GCN is employed and the representations from different spaces are simply concat at this stage, benefit from the hierarchical context of events, resulting in an improvement of 2.42% on multimodal setting and 1.75% on unimodal setting respectively (3 rd row).Then DSI is introduced to facilitate information interactions, which further enhances the performance on multimodal input setting by 1.15% and unimodal input setting by 2.31% (5 th row). 2) Effect of LSHAD. LSHAD is a crucial element of our HE-GCN for constructing the message graph. In Table 2 (2 nd and 3 rd rows), we present experiments comparing HGCN used in HyperVD [25] , which employs a hard node selection strategy (nodes selected by a fixed threshold), with HE-GCN, which uses our introduced layer-sensitive hyperbolic association degrees for node selection in message aggregation. The performance of AP improved by 1.45% in the multimodal setting and by 1.77% on the unimodal setting. These results demonstrate that our LSHAD is a better strategy and helps to capture the hierarchical context of events. 3) Lorentzian metric in DSI. As shown in Table 2 (4 th and 5 th rows), we also explore the effect of cosine similarity and Lorentzian metric in our DSI, and the results show that using cosine similarity to calculate the attention between nodes resulted in a 0.7% lower performance compared with using the Lorentzian metric. This indicates that the Lorentzian metric is more effective in measuring feature similarity across different spaces.Feature Discrimination Visualization. We present t-SNE visualizations of feature distributions on the XD-Violence test set. As shown in Figure 3 , red dots represent violent features, while purple dots represent normal features. The clear clustering of violent and non-violent features demonstrates the effectiveness of DSRL.2) Qualitative Visualizations. We illustrate the qualitative visualizations of VVD for the test video from the XD-Violence dataset. Figure 4 shows that DSRL can accurately detect violent events and has a better detection performance than the baseline.3) Qualitative Visualizations of DSRL in the Context of Ambiguous Violence. We put up qualitative visualizations of DSRL when handling ambiguous violence. Figure 5 demonstrates that DSRL effectively resolves ambiguous violence, which single-space representation learning struggles with.In Figure 5 (a), the first frame shows smoke caused by fire. Euclidean space representation, relying on visual features, misidentifies the smoke as violence. Hyperbolic space representation, considering contextual information, also misidentifies it due to preceding violent frames. DSRL, however, combines both perspectives: Euclidean space flags the smoke as a potential violence indicator, while hyperbolic space recognizes the fire context. This integration allows DSRL to accurately classify the smoke as non-violent. This supports our motivation: hyperbolic representation enhances hierarchical event relations but weakens visual feature expression, while Euclidean representation emphasizes visual features but overlooks event relationships. DSRL effectively addresses ambiguous violence, which is challenging for either space alone. More performance visualizations are included in the Appendix. Our designs, HE-GCN and DSI, are specifically crafted to effectively capture hierarchical contextual information of events and to integrate two distinct spaces of information, respectively. To evaluate the benefits of these designs, we conducted experiments on the XD-Violence dataset, training three models for 30 epochs each on a single NVIDIA RTX A6000 GPU: the Baseline (GCN), Baseline+HE-GCN, and Baseline+HE-GCN+DSI (referred to as DSRL). As shown in  
conclusions: 
 In this paper, we propose a comprehensive geometric representation learning method, Dual-Space Representation Learning (DSRL) which integrates the benefits of Euclidean and hyperbolic geometries to improve the discrimination of ambiguous violence. Hyperbolic Energy-constrained Graph Convolutional Network (HE-GCN) is designed to better capture the hierarchical context of events. Additionally, Dual-Space Interaction (DSI) is designed to facilitate information interactions. Our method achieves SOTA performance on the XD-Violence dataset in both unimodal and multimodal settings, especially excelling in resolving ambiguous violence.Limitions. Our DSRL is effective for VVD in a multimodal input setting. However, DSRL only utilizes basic audio information, potentially overlooking the more detailed semantic content present in the audio. How to further narrow this limitation is our future research focus. Due to the bias issues in the XD-Violence and UCF-Crime datasets, which fail to adequately represent diverse backgrounds, the fairness and generalization of the model may be affected. In the future, we hope to use more diverse datasets and conduct bias analysis to prevent the model from producing unfair outcomes due to false associations with gender or race.Theorem 1. ∀x ∈ L n , M ∈ R (m+1)×(n+1) , we have f x (M)x ∈ L m K . Proof 1. y = f x (M)x = √ ∥W x∥ 2 -1/K v ⊤ x v ⊤ x Wx = ∥W x∥ 2 -1/K Wx ⟨y, y⟩ L = -(∥W x∥ 2 -1/K) + (Wx) ⊤ Wx = 1 K -∥W x∥ 2 + ∥W x∥ 2 = 1 KThus, f x (M)x lies on the manifold L of the Lorentz model.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: We provide detailed information on the experimental setup in the implementation details section of the paper. See Sec. C in the Appendix.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: Please see Sec. 5 and Sec. C.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: Please see Sec.F.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [No] Justification: The used datasets are publicly available, and widely used in existing VVD methods. Thus, we do not mention the license of the dataset.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. Justification: The datasets used in this paper, XD-Violence and UCF-Crime, are publicly available and widely used in research. These datasets were collected by their original creators and made accessible for research purposes. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? Answer: [No] Justification: Since we are using datasets that are already publicly available and have been extensively used in previous research, and given that the content does not involve sensitive personal information, this study did not undergo an independent IRB review. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: As depicted in Figure 2 , the entire process can be divided into feature preprocessing to integrate the features of the two modalities; representation learning in hyperbolic space to learn the hierarchical context of events; representation learning in Euclidean space to learn the visual features of events; interaction between the two spaces to promote cross-space enhancement; and finally, feeding the features into a hyperbolic classifier for classification.Feature Preprocessing. Following the previous works [34, 23] , the visual and audio segments are processed by the I3D [2] network pretrained on the Kinetics-400 dataset and the VGGish [11] network pretrained on a large YouTube dataset, respectively. After that, we perform further feature extraction using simple convolution and pooling operations. A simple cross-modal attention mechanism is then employed to enhance the audio features, which are subsequently concatenated with the visual features to form the fused features.Hyperbolic Representation Learning. In this part, we first use HE-GCN to learn the hierarchical context of events. Meanwhile, the temporal relation is also crucial for numerous video-based tasks. Therefore, we construct a temporal relation graph directly based on the temporal structure of a video and learn the temporal relation among snippets in hyperbolic space via HGCN. Its adjacency matrix A T ∈ R T ×T is only dependent on temporal positions of the i-th and j-th snippets, which can be defined as:EQUATIONwhere σ controls the range of influence of distance relation. Finally, we use HE-GCN for semantic message aggregation and HGCN for temporal message aggregation, then concatenate them to obtain a hyperbolic space representation.Euclidean Representation learning. The process of representation learning in Euclidean space is similar to that in hyperbolic space and follows a dual-branch structure, considering both semantic and temporal relationships. We use cosine similarity to compute the semantic similarity in Euclidean space and use Eq. 21 to compute temporal relationships. Finally, we employ GCN for message aggregation and concatenate them to obtain the final representation in Euclidean space.Dual-Space Interaction. In this part, we primarily use cross-space attention to enhance the interaction between the features of the two spaces. The detailed process is described in Sec. 4.2.Hyperbolic Classifier. As shown in Figure .2, we input the enhanced embeddings from DSI into hyperbolic classifier utilizing Lorentzian metric, which can be formalized as:EQUATION)where σ is sigmoid function and W is weight matrices. b and ϵ denotes bias term and hyper-parameter, respectively. Lastly, we supervise the training of the violence scores obtained by the model with the real labels using the binary cross-entropy loss.Dataset. We conducted experiments on XD-Violence with both multi-modal input settings and singlemodal input settings. Additionally, we performed experiments on UCF-Crime with single-modal input settings to demonstrate the generalization capability of DSRL. For the audio data, we first divide each audio into 960-ms overlapped segments and compute the log-mel spectrogram with 96 × 64 bins. Our proposed method is trained for 30 epochs in total, and the batch size is 256. The initial learning rate is 0.001, which is dynamically adjusted by a cosine annealing scheduler [13] . We use Adam [14] as the optimizer without weight decay. For hyper-parameters, we set β as 0.8, γ as 1.2, α as 0.3, and dropout rate as 0.6. Following [25] , σ is empirically set to e. For the MIL, we set the value k of k-max activation as T 16 + 1 , where T denotes the length of the input feature. Experiments Compute Resources. We use an Intel(R) Xeon(R) Platinum 8260 CPU @ 2.40GHz, a NVIDIA RTX A6000 GPU to conduct experiments. We use CUDA 12.2, Python 3.9.16, and Pytorch 1.12.1.Reasons for the design choices in LSHAD with its multiple hyperparameters and threshold criteria.Inspired by the Global-first principle [4] that humans always have cognition on global first and then focus on local, we propose a novel node selection strategy, which guarantees the model captures the broader global context first with a relaxed threshold at the beginning of message aggregation and then focuses on the local context with more strict thresholds. To achieve this, we introduce the LSHAD construction rule, which calculates an LSHAD threshold based on the number of the current layer K and hyperbolic Dirichlet energy of the current layer. As the K increases and the hyperbolic Dirichlet energy decreases, the LSHAD threshold increases and is limited to 0 and 1 by the sigmoid function. If there is no β and γ, the threshold in the first layer will be strict (> 0.5), causing the overlook of some global context information. Therefore, to make our node selection threshold conform to the Global-first principle, we introduced the two hyperparameters in LSHAD, where β controls the influence of the number of current layer k and γ acts as a bias to fine-tune the threshold. Moreover, we conducted an ablation study to determine the optimal value of the two hyperparameters (β,γ), where β ranges from [0.2,0.4,0.6,0.8,1.0] and γ ranges from [1.0,1.2,1.4,1.6,1.8,2.0]. The results in the table below reveal that when γ -β is 0.4, the performance is optimal, so we chose a pair (0.8, 1.2) from this set. Ablation studies on some hyperparameters in DSI. Moreover, we conduct ablation studies on some hyperparameters in DSI. Inference visualizations of the ablation modules. In this section, we have supplemented inference visualization results for the ablation module, and conducted corresponding visualisation experiments to explore the discriminative power of the model w/ or w/o our core modules, HE-GCN and DSI. We conducted the analysis from two dimensions (the feature-level and the frame-leve), and the results are shown in Figure 6 and Figure 7 .At the feature-level, the results are shown in Figure 6 . Compared to GCN, HE-GCN can capture the hierarchical context of events, effectively separating features. This results in a greater distance between feature clusters compared to Figure 6 Meanwhile, at the frame-level, experiments conducted on two test videos as shown in Figure 7 demonstrate that our method significantly improves the discriminative power for identifying violent frames compared to the baseline, which uses only GCN. Compared with the model w/o our core modules, both HE-GCN and DSI contribute to detecting violent frames. Moreover, we provide more qualitative results, including the qualitative visualizations of VVD (Figure 8 ) and the qualitative visualizations of DSRL in the context of ambiguous violence (Figure 9 ).Figure 8 illustrates that DSRL can accurately distinguish between violent and normal events, demonstrating the effectiveness of DSRL. Additionally, compared with the baseline curves, our method shows better performance, further validating the effectiveness of the modules we designed. Qualitative Visualizations of DSRL in the context of ambiguous violence. Figure9 shows that in the XD-Violence test set, DSRL accurately detects the correct categories of ambiguous violence, while using only Euclidean space or only hyperbolic space fails to correctly detect these instances.Potential positive societal impacts.Our work can be more effective in identifying incidents of violence, albeit it may be ambiguous. This contributes to a higher level of safety in the community and the public's sense of security. Potential negative societal impacts.If our algorithm were to be widely applied in monitoring and law enforcement domains, it could  


Decision: The paper proposes a method for violence detection from videos. The reviewers found the method interesting, novel and effective. The evaluation shows the effectiveness of the proposed method. The authors rebuttal further clarified questions from the reviewers. On the other hand, it is very important to acknowledge and discuss the issue of potential model bias. For example, the learned model may work better just by taking advantage of false association between gender/race and violence. This has been shown in many related works, such as racial bias in hate speech detection. It is very strongly recommended that the authors discuss this issue in details and also perform additional analysis showing the model (or the improvement) is not biased against protected attributes.

Reviews: review: paper_summary: paper_summary: The paper introduces a novel method called Dual-Space Representation Learning (DSRL) aimed at enhancing the detection of video violence, particularly in scenarios where the violence is weakly supervised and visually ambiguous. 

DSRL combines the strengths of both Euclidean and hyperbolic geometries to capture discriminative features for violence detection, leveraging the hierarchical structure modeling capability of hyperbolic spaces. 

Two specialized modules are designed: the Hyperbolic Energy-constrained Graph Convolutional Network (HE-GCN) module and the Dual-Space Interaction (DSI) module, enhancing the understanding of event hierarchies and promoting dual-space cooperation.

Comprehensive experiments on the XD-Violence dataset demonstrate the effectiveness of DSRL, outperforming existing methods in both unimodal and multimodal settings.
strength_weakness: strength_weakness: Subjectivity: The Layer-Sensitive Hyperbolic Association Degree (LSHAD) proposed by the author contains multiple hyperparameters, and LSHAD is used to examine the threshold of the message graph. It is very important. There is a lack of explanation for why it is designed in this way.

Complexity: DSRL combines Euclidean and hyperbolic geometric spaces, as well as cross-space interaction mechanisms, which may increase the complexity of the model, resulting in higher demands on computing resources and training time. 

Parameter sensitivity: DSRL contains multiple hyper-parameters, such as β, γ, α, etc. The selection of these parameters may have a significant impact on model performance. This article lacks an analysis of parameter adjustment.

Application-specific limitations: DSRL is specifically designed for the task of video violence detection, which may mean that further adjustments or optimizations are required when applying it to other types of video content analysis tasks.
questions: questions: 1. What are the specific reasons for the design choices in the Layer-Sensitive Hyperbolic Association Degree (LSHAD) with its multiple hyperparameters and threshold criteria? Was there any experimental or theoretical analysis conducted to support these design decisions?
2. Considering the increased demand for computing resources and training time, are the benefits of these complex designs justified?
3. What methods or heuristic approaches are used to determine the optimal values of these hyperparameters?  Has a systematic sensitivity analysis been conducted for the selection of hyperparameters (such as β, γ, α, etc.) in the DSRL model?
4. Has the DSRL model been tested on other types of video content analysis tasks? If so, how did the model perform in these tasks? Are there any additional adjustments or optimizations required to adapt the model to different video content analysis applications?
limitations: limitations: The authors discussed their limitations.
paper_summary: paper_summary: This paper proposes leveraging dual-space learning, encompassing both Euclidean and Hyperbolic spaces, to enhance discriminative capacity by capitalizing on the strengths inherent in hyperbolic learning. To overcome the limitations of the hard node strategy in the previous method, this work introduces DSRL (Dual-Space Representation Learning), which enhances node aggregation selection by utilizing layer-sensitive hyperbolic association degrees constrained by hyperbolic Dirichlet energy. A cross-space attention mechanism is then proposed to facilitate information interactions between Euclidean and hyperbolic space to capture better discriminative features.
strength_weakness: strength_weakness: ### Experiments:
It would have made the manuscript more convincing if the authors could provide inference visualizations of the ablation modules, which means to better illustrate the discriminative power of the method w/ or w/o your core modules.

### Minor Typos:
- Line 137:  $\mathcal{L}^{\/}$ should be $\mathcal{L}^{n}$.
- Eq (17) misses the right bracket for the $softmax$ function.
- In Eq (9), it misses the symbol annotations for $\mathbf{v}$. Did you use the hyperbolic transformation strategy proposed in "Fully hyperbolic neural networks", if so, the citation is missed here.
questions: questions: - Why utilize these two spaces (hyperbolic and Euclidean) for the representation learning? Have you explored different hyperbolic spaces such as Lorentz and Poincaré Ball?
- How about the model's training stability?
limitations: limitations: Please refer to the weakness.
paper_summary: paper_summary: This paper presents a novel approach called Dual-Space Representation Learning (DSRL) for weakly supervised Video Violence Detection (VVD). Traditional VVD methods rely heavily on Euclidean space representation, which often fails to distinguish between visually similar events. The proposed DSRL method leverages both Euclidean and hyperbolic geometries to enhance the discriminative capacity of features. It introduces two key modules: the Hyperbolic Energy-constrained Graph Convolutional Network (HE-GCN) and the Dual-Space Interaction (DSI) module, to facilitate better information interactions and improve violence detection accuracy. The method achieves state-of-the-art performance on the XD-Violence dataset in both unimodal and multimodal settings. Additionally, the method shows good effectiveness on the UCF-Crime dataset, further proving its strong generalization ability across datasets.
strength_weakness: strength_weakness: 1.For the HE-GCN module, I am curious about the relationship between HDE and LSHAD. The authors should explain the relationship between HDE and LSHAD in detail?

2.Figure 2 should be revised to clarify that the method is applicable to both unimodal and multimodal inputs, not just multimodal. Currently, it may mislead readers.

3.The Preliminaries section is well-detailed and informative, but it may be complex and difficult for a broader audience to understand.

4.There are some minor writing issues: Lines 439 and 479, “Figure” and “Fig” should be standardized; Line 147, there is a symbol display error.
questions: questions: Same as weakness.
limitations: limitations: N/A
paper_summary: paper_summary: This paper presents a pioneering approach called Dual-Space Representation Learning (DSRL) for the task of weakly supervised video violence detection.  The overall framework contains the Hyperbolic Energy-constrained Graph Convolutional Network (HE-GCN) for capturing event hierarchies and the Dual-Space Interaction (DSI) module to facilitate cross-space feature integration. Experiments on XD-Violence and UCF-Crime datasets show the advantage of the proposed method.
strength_weakness: strength_weakness: - **Novelty**: Some modules in this paper lack innovation, such as the cross-graph attention mechanism is commonly used in previous works.
- **Model Complexity**: This paper does not discuss the computational complexity or parameters of the DSRL model. For practical applications, especially in real-time surveillance scenarios, it's important to ensure that the model can operate efficiently with minimal latency.
questions: questions: - **Qualitative Visualizations**: Does the 'Hyperbolic' in Figure 5 represent the result of HyperVD? If not, I wonder whether HyperVD can handle ambiguous violence.
limitations: limitations: Please see #Weaknesses and #Questions.

Comments: title: Comments from Reviewer rxrS
new comments
comment: Thank you for the authors' response, most of my concerns have been addressed, and considering the application area would be somewhat domain specific, I will stay with my original rating, i.e. being slightly positive.
Thank you for the valuable feedback. We would like to address the ethical review concerns as follows：  
    **Dataset Sources:** The datasets used in this paper include XD-Violence[1] and UCF-Crime[2]. Both of these datasets are publicly available and have been widely used in the research community.   
    **Acquisition and Use of Datasets:** These datasets were obtained from their respective official websites and were used with appropriate authorization as provided by the dataset creators. We have only utilized the publicly available portions of these datasets for academic research purposes, adhering strictly to the usage terms set by the dataset providers.  
    **Privacy Concerns:** The XD-Violence and UCF-Crime datasets primarily contain video clips from public settings and do not involve personal sensitive information or privacy data. Therefore, we believe the use of these datasets does not pose any privacy concerns.  
    **IRB Review:** Since we are using datasets that are already publicly available and have been extensively used in previous research, and given that the content does not involve sensitive personal information, this study did not undergo an independent IRB review.  
**Subject Consent:** The datasets used in this paper, XD-Violence and UCF-Crime, are publicly available and widely used in research. These datasets were collected by their original creators and made accessible for research purposes. As secondary users, we do not have direct information about whether individuals in the videos provided explicit consent. We rely on the dataset creators' adherence to ethical standards during data collection. We will address this in the limitations section, acknowledging the potential ethical implications and emphasizing the importance of consent in future data collection.     
   **Dataset Biases:** We acknowledge the potential for biases in these datasets, as they may not fully represent the diversity of violent incidents across different contexts, cultures, or demographic groups. The XD-Violence and UCF-Crime datasets may have inherent biases related to the selection of video content, such as over- or under-representation of certain types of violence or specific geographical locations. These biases could affect the generalizability of our model’s performance.   

Future work will focus on addressing these biases by incorporating more diverse datasets and employing techniques to mitigate their effects. 

In response, we will make the following improvements in our paper:

1. We will add explanations regarding the **Dataset Sources**, **Acquisition and Use of Datasets**, **Privacy Concerns**,**Subject Consent** and **IRB Review** to the dataset introduction section in the paper's appendix. These additions will address ethical concerns related to the data used in our study, ensuring transparency about how the datasets were sourced, the authorization obtained for their use, the handling of privacy issues, the consideration of subject consent, and the rationale for not undergoing an independent IRB review. 

2. We will add a discussion on **Dataset Biases** in the limitations section of the paper. Specifically, we will discuss the potential biases present in the XD-Violence and UCF-Crime datasets, such as the possibility that these datasets may not fully represent the diversity of violent incidents across different backgrounds, cultures, or demographic groups. These biases could lead to the over- or under-representation of certain types of violence or specific geographical locations, which could, in turn, affect the generalizability and fairness of the model. We will emphasize this point and suggest that future research should address these biases by using more diverse datasets and conducting more rigorous testing to ensure the fairness and generalizability of the model. 

We believe that the data usage in this paper aligns with ethical standards for academic research. We hope our responses have addressed your concerns. If you have any further questions or comments, please kindly let us know, and we will be glad to respond. 

[1] Peng Wu, Jing Liu, Yujia Shi, Yujia Sun, Fangtao Shao, Zhaoyang Wu, and Zhiwei Yang. Not only look, but also listen: Learning multimodal violence detection under weak supervision. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16, pages 322–339. Springer, 2020.  
[2] Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6479–6488, 2018.
Thank you for reviewing our paper and for your valuable suggestions. Here are our responses to the ethical concerns you mentioned about our paper.  
**Demographic Information of the Datasets:** The datasets we mentioned, XD-Violence and UCF-Crime, are publicly available and commonly used in the VVD task. Besides, the original published papers [1][2] do not provide detailed demographic information. 
Following your suggestion, we will note this in the limitations section and discuss how the lack of such information might affect the model's performance, particularly for underrepresented groups. 

**Model Performance in Different Contexts:** Although XD-Violence and UCF-Crime datasets enhance diversity by collecting videos across multiple languages and cultural backgrounds, which may partially mitigate performance biases in different contexts, we acknowledge that the model may still exhibit false positives and false negatives in environments significantly different from the training data. We'll address this in the limitations section, advising caution when applying the model in new settings and recommending further localized testing. 

**Potential Disparate Impact on Protected Groups:** While the diversity of the XD-Violence and UCF-Crime datasets helps reduce bias, it may not fully eliminate unfair impacts on protected groups. We acknowledge the following limitations: 

1. Certain groups may still be underrepresented, leading to potential underperformance for these groups.

2. The dataset's scenes and environments may not fully represent all real-world settings, potentially causing suboptimal performance in specific contexts.

These issues will be addressed in the limitations section, with recommendations for future research to validate the model's fairness and generalization through more extensive testing and dataset expansion.

In response, we will make the following improvements in the limitations section of our paper:

   * We will add content to explain the lack of detailed demographic information in the XD-Violence and UCF-Crime datasets, discussing the potential biases this may introduce for underrepresented groups.
   * We will highlight that although our datasets reflect some cultural and linguistic diversity, they do not cover all real-world environments, and thus the model may produce false positives or negatives in different settings. We will recommend thorough testing in specific application scenarios before deployment.
   * We will acknowledge that despite the datasets covering various skin tones, ages, and cultural backgrounds, there may still be unfair impacts on certain protected groups due to underrepresentation. Future research will be encouraged to incorporate more diverse datasets and conduct targeted bias analysis.
 
We hope our responses have addressed your concerns. If you have any further questions or comments, please kindly let us know, and we will be glad to respond. 

[1] Peng Wu, Jing Liu, Yujia Shi, Yujia Sun, Fangtao Shao, Zhaoyang Wu, and Zhiwei Yang. Not only look, but also listen: Learning multimodal violence detection under weak supervision. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16, pages 322–339. Springer, 2020.  
[2] Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6479–6488, 2018.
Thank you for the authors' detailed response, most of my concerns have been addressed and I maintain my original rating.
Thanks for your feedback! I will keep my original rating.
Thank you for the response. 

Following your suggestion, we have evaluated the real-time processing speed of HyperVD and MACIL-SD with an NVIDIA RTX A6000 GPU for a fair comparison, and the FPS of HyperVD and MACIL-SD is 98.90 and 102.99, respectively. 

In this work, we focus on addressing the problem of ambiguous violence. To tackle this issue, we have to introduce some additional computations, with the speed sacrificed to some extent. Despite our method not being as fast as these two methods, it still meets real-time processing requirements and is acceptable. Moreover, our AP performance on the XD-Violence dataset has improved by 4.21\% over MACIL-SD and 1.94\% over HyperVD. Notably, from the visualization comparison experiments, as shown in Figure 5 ("Hyperbolic" is HyperVD) of the submitted manuscript, our method is obviously better than HyperVD in handling ambiguous violence.

We hope our responses have addressed your concerns. If you have any further questions or comments, please kindly let us know, and we will be glad to respond.
Thanks for providing the response. It partially addressed my concerns. However, can you provide the real-time processing speed of HyperVD and MACIL-SD? It would be nice to see some comparisons between them.
The authors have addressed all my concerns, so I stay with my original score.