Cefdet: Cognitive Effectiveness Network Based on Fuzzy Inference for Action Detection

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 09:53:30

Abstract: Action detection and understanding provide the foundation for the generation and interaction of multimedia content. However, existing methods mainly focus on constructing complex relational inference networks, overlooking the judgment of detection effectiveness. Moreover, these methods frequently generate detection results with cognitive abnormalities. To solve the above problems, this study proposes a cognitive effectiveness network based on fuzzy inference (Cefdet), which introduces the concept of “cognition-based detection” to simulate human cognition. First, a fuzzy-driven cognitive effectiveness evaluation module (FCM) is established to introduce fuzzy inference into action detection. FCM is combined with human action features to simulate the cognition-based detection process, which clearly locates the position of frames with cognitive abnormalities. Then, a fuzzy cognitive update strategy (FCS) is proposed based on the FCM, which utilizes fuzzy logic to re-detect the cognition-based detection results and effectively update the results with cognitive abnormalities. Experimental results demonstrate that Cefdet exhibits superior performance against several mainstream algorithms on public datasets, validating its effectiveness and superiority.

Main paper: 
 introduction: 
 Multimedia is an essential branch of modern information technology, which comprehensively processes various media forms of information such as text, images, audio, and video through computers or other electronic devices. With the improvement of computer processing capabilities, the creation and sharing of multimedia content have been further promoted. Multimedia technology has driven the innovation of content presentation and provided a wealth of application scenarios for artificial intelligence research [1, 2] .Action detection is widely applied in multimedia analysis. It focuses on identifying specific human actions or behavior patterns from videos and can be combined with audio signals or other types of sensor data for multimedia analysis. In recent years, the accuracy of action detection has significantly improved with the advent of deep learning, particularly convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Action detection is gradually playing a crucial role in various applications, including smart security, health monitoring, and human-computer interactions [3, 4] .However, existing methods face challenges in determining the effectiveness of detection results. Additionally, existing methods constantly generate detection results with cognitive abnormalities due to ineffective judgment. As shown in Figure 1 (a), actions such as "pick" and "sit" exhibit high similarity. Existing methods wrongly detect "pick" as "sit" while affecting the action detection of subsequent frames. In addition, as depicted in Figure 1 (b), action changes from "swing_baseball" to "shot_gun" do not conform to human action norms. Existing methods detect two unrelated actions as adjacent and recognize one or several consecutive frames as a complete action, which is inconsistent with human cognition. These problems hinder the further application of action detection methods.Therefore, this study proposes a cognitive effectiveness network based on fuzzy inference (Cefdet). It introduces the concept of "cognition-based detection" and combines features in human action with fuzzy inference to simulate the cognition-based detection process. The effectiveness of the detection results is accurately determined using cognition-based detection. Moreover, the detection results are divided based on their effectiveness, and fuzzy logic is employed to dynamically update the detection results with lowlevel cognition, which repairs the detection results with cognitive abnormalities.The contributions of this study are summarized as follows:• This paper proposes the FCM for detecting the effectiveness of frames. It takes the features of human actions, including the confidence of each frame, the correlation between adjacent frames, and the position score of each frame, as inputs to the fuzzy inference system. This simulates a cognitionbased detection process to obtain effectiveness, thus accurately locating the position of action frames with cognitive abnormalities.  
related work 2.1 video understanding: 
 Video understanding includes identifying the activities that occur during the editing process. Typically, the time span for editing is a few seconds, and there is only one annotation. Most early video comprehension methods [5, 6] utilized 2D image CNNs and introduced long-short-term memory (LSTM) [7] to learn video time structures. Subsequently, various 3D CNNs [8] are proposed for video understanding because they process the entire video clip as input rather than treat it as a frame sequence [9] . Due to the scarcity of tagged video datasets, several researchers have relied on a model pre-trained on ImageNet and used it as the backbone for video feature extraction. Some studies [10, 11] decoupled 3D convolution into 2D space and 1D time kernels to reduce the model size. Twostream networks [12] constitute another widely employed video understanding method. Due to its ability to handle only a small portion of the input frames, it achieved a superior balance between accuracy and complexity. Cheng et al. [13] proposed a new method to recover the intermediate features between two sparse samples and adjacent video frames, achieving significant results.Action detection has received increasing attention from researchers as an essential technology for video understanding [14] . Owing to the potential of multiple actions within each frame, it is necessary to detect the actions of individual entities within the current frame rather than categorizing the entire video into a single class. Inspired by deep CNNs for object detection [15] , methods based on action detection typically apply 2D position anchors or offline object detectors to keyframes to locate human subjects. Subsequently, they focused on improving action detection and incorporating temporal patterns by leveraging the optical flow for additional flow fusion. Some methods [16, 17] have applied 3D convolutional networks to capture temporal information for identifying actions and have achieved excellent results. Feichtenhofer et al. [14] proposed a slow network that can better capture spatiotemporal information.Recent research on spatiotemporal action detection has emphasized modeling the interaction between classified individuals and their environment. Recent methods [18, 19] proposed explicitly modeling the relationships between actors and objects. A dualmode [20] interaction structure is constructed based on human posture, hands, and objects, effectively improving the accuracy of action detection.Fuzzy inference is widely used in computer vision applications. It is a core component of fuzzy logic and involves the application of fuzzy sets and rules to input data to obtain fuzzy output results. Fuzzy inference performs better in handling uncertain information and can more reasonably describe the actual needs and achieve intelligent control. Several studies based on fuzzy inference have been proposed. For example, an improved fuzzy clustering-based classifier [21] employs 𝐿 2 regularization to mitigate overfitting. This classifier demonstrated remarkable performance in various classification tasks. Eyoh et al. [22] introduced an interval-based intuitionistic fuzzy system with membership and non-membership functions for identification and prediction. Multi-layer interval Type-2 fuzzy limit machine learning (ML-IT2-FELM) [23] identify walking activities and used wearable sensors to determine gait. Fuzzy learning is utilized to obtain appropriate results for the activities. Cao et al. [24] integrated interval Type-2 fuzzy sets into a fuzzy rough neural network to forecast stock time series. In addition, other studies [25, 26] introduced a common quadratic Lyapunov function to analyze the stability and design controllers for fuzzy closed-loop systems. 
proposed method 3.1 overall framework: 
 Due to the uncertainty of daily human activities, existing algorithms cannot effectively determine the effectiveness of their detection results. Moreover, the lack of effective judgment frequently leads to detection results with cognitive abnormalities in existing algorithms. To solve these problems, this study introduces the concept of "cognition-based detection" into action detection. First, fuzzy inference is employed to simulate the cognition-based detection process by combining human action features to accurately locate the positions of frames with cognitive abnormalities. Then, the local features of the frames with cognitive abnormalities are dynamically fused based on fuzzy logic for re-detection. The effectiveness before and after re-detection is used to repair detection results inconsistent with cognition.The overall framework of Cefdet is illustrated in Figure 2 . Initially, the FCM is designed after the video frames pass through a network. It evaluates the effectiveness of each frame by considering the confidence, correlation between adjacent frames, and position score, dividing the frames into low and high levels. Subsequently, the FCS is proposed to update the frames with low-level cognition. Specifically, it constructs correlated sequences using frames with high-level cognition. The features are then weighted based on the similarity between the correlated sequences and frames with lowlevel cognition for fusion, which is utilized for re-detection. Finally, the decision to update the results is made based on the effectiveness of the action frames before and after the re-detection.Due to the lack of effective judgment of detection results, existing action detection algorithms frequently obtain detection results with cognitive abnormalities in complex scenes. The use of confidence scores to evaluate the effectiveness of an action frame is not comprehensive, and more factors should be considered. Therefore, this study proposes the FCM that cooperatively judges each frame's effectiveness by combining each frame's confidence, the correlation between adjacent frames, and the position score of each frame with a fuzzy inference system. Fuzzy logic is employed to simulate the cognition-based detection process to accurately determine the effectiveness of each frame and locate the positions of frames with cognitive abnormalities. FCM is a fuzzy inference engine with four components: feature quantization, fuzzification, fuzzy inference, and defuzzification. Brief descriptions of each component are provided below.Feature quantification. The objective of fuzzy inference is to assess the effectiveness of the frames after detection. Therefore, it is essential to construct feature vectors of the frames as inputs for fuzzy inference. For the input video sequence 𝐼 𝑖 , the feature vector consists of diverse feature values, including the confidence of each frame 𝐶 𝑖 , correlation between adjacent frames 𝑁 𝑖 , and position score of each frame 𝐺 𝑖 . 1. Confidence. As an important evaluation criterion, the confidence of the action detection network is expressed in the following formula.EQUATIONwhere 𝜑 ( * ) refers to the action detection network in [20] , which is tasked to predict the true value from 𝐼 𝑖 . 2. Correlation between adjacent frames. In video sequences, action sequences with short intervals have rich correlations, and the evolution from one action to another is a gradual process. It is possible to determine whether the subsequent action is reasonable based on a previous action. Therefore, the positions of action frames that do not conform to human action norms are accurately located using this correlation. This study employs normalized pointwise mutual information (NPMI) to measure the correlation between actions. NPMI is defined as follows:EQUATIONwhere 𝐾 𝐼 𝑖 refers to the action category of the i-th frame; 𝑃 𝐾 between the two actions, 𝑁 𝑖 = 0 indicates independence between the two actions, and 𝑁 𝑖 = -1 denotes that the two actions have never appeared simultaneously. Figure 3 shows the correlation between the different actions calculated by the NPMI.3. Position score. Action detection is the process of observation and refinement with a clear time boundary between the beginning and end of an action. The position of the current action frame is noted. If the frame is at the center of the current action, it is considered highly reliable. By contrast, if the frame is at the boundary position of the current action, the reliability of the frame requires further detection. In this study, the reliability of a frame is determined by comparing its corresponding position within a complete action. For action category 𝐾 𝐼 𝑡 at frame𝑡, if 𝐾 𝐼 𝑡 -1 ≠ 𝐾 𝐼 𝑡 , 𝐾 𝐼 𝑡 = 𝐾 𝐼 𝑡 +1 = • • • = 𝐾 𝐼 𝑡 +𝑛 and 𝐾 𝐼 𝑡 ≠ 𝐾 𝐼 𝑡 +𝑛+1 , then [𝐼 𝑡 , • • • , 𝐼 𝑡 +𝑛 ]can be considered as a complete action. The reliability of this frame is defined as follows:EQUATIONEQUATIONwhere 𝐺 𝑖 is the position score of the i-th frame; 𝜇 and 𝜎 are the mean and standard deviation of the Gaussian distribution, respectively;𝜇 = 𝑡 + 𝑛 2 , and 𝜎 = √︃ 𝑡 +𝑛 𝑖=𝑡 (𝑖 -𝜇 ) 2 𝑛. Figure 4 shows the position scores of the frames in the complete action sequence.Fuzzification. The five-level fuzzification rule 𝑘 = {𝑁 𝐵, 𝑁 𝑆, 𝑍𝑂, 𝑃𝑆, 𝑃𝐵} is employed to determine the degree of membership of the feature vector to a fuzzy set: negative big (𝑁 𝐵), negative small (𝑁 𝑆), zero (𝑍𝑂), positive small (𝑃𝑆), or positive big (𝑃𝐵). ℎ 𝑖 = {𝐶 𝑖 , 𝑁 𝑖 , 𝐺 𝑖 , 𝑈 𝑖 } is the training set of fuzzy inference, where 𝑈 𝑖 = 𝜇 1 𝐶 𝑖 + 𝜇 2 𝑁 𝑖 + (1 -𝜇 1 -𝜇 2 ) 𝐺 𝑖 , the universe is defined as 𝑃 = {𝑃 (ℎ 𝑖 )}, 𝜇 𝑘 𝑃 (ℎ 𝑖 ) indicates the membership degree of ℎ 𝑖 to the set 𝑘 𝑃 , and 𝐹 ℎ 𝑖 = 𝜇 𝑘 𝑃 (ℎ 𝑖 ) reports that ℎ 𝑖 belongs to the fuzzy set of universe 𝑃, as illustrated below:EQUATION)Figure 4 : Position score of frames in action sequences. In a complete action sequence, the closer to the center the frame is, the more reliable it is. Fuzzy inference. Because the fuzzy inference system had three input variables, driving 125 fuzzy rules. Table 1 shows examples of fuzzy rules. The fuzzy set generated by each fuzzy rule 𝜔 𝑖,𝑗 is obtained by performing the combination of membership degrees on the front parts of each rule, where 𝑗 ∈ [1, 2, • • • , 𝑤] is the number of active rules. Subsequently, the conclusion of fuzzy inference Û𝑖 is obtained through a disjunction operation. The formula is as follows:EQUATION)EQUATIONwhere 𝑈 𝑗 𝑖 is the j-th element in the output universe 𝑃 (𝑈 𝑖 ), and 𝐹 𝑗 𝑈 𝑖 ∈ 𝑘 𝑃 (𝑈 𝑖 ) denotes any fuzzy set on 𝑃 (𝑈 𝑖 ).Defuzzification. During defuzzification, the centroid method is selected to obtain the output. The abscissa value corresponding to the membership function curve of the fuzzy set and the center of the area surrounded by it is the effectiveness of frame 𝑢 𝑖 , which can be defined as follows:EQUATIONwhere 𝜇 𝑘 𝑃 (𝑈 𝑖 ) 𝑈 𝑗 𝑖 is the membership degree of 𝑈 𝑗 𝑖 to set 𝑘 𝑃 (𝑈 𝑖 ) , and𝑃 (𝑈 𝑖 ) = 𝑈 1 𝑖 , 𝑈 2 𝑖 , • • • , 𝑈 𝑗 𝑖 .Figure 5 shows the overall framework of FCM. First, the corresponding feature vectors are extracted from the action detection and are fuzzified into membership degrees of different sets in the respective universe. Fuzzy logic is then employed to aggregate the active fuzzy rules. Finally, the effectiveness of the frame is obtained through defuzzification.The FCM evaluates the effectiveness of each frame by incorporating multiple perspectives. This aids in the accurate localization of the detection results with cognitive abnormalities and facilitates the subsequent stages of further detection.The detection performance for each frame in a video sequence depends on the effectiveness of the surrounding action frames. The poor effectiveness of the surrounding action frames often leads to noise interference in detecting the target frame. Therefore, frames are divided into low and high levels based on their effectiveness.Frames with high-level cognition are considered reliable predictions, whereas the remaining frames require further detection as follows:EQUATIONSubsequently, a storage space is constructed to store correlated sequences composed of frames with high-level cognition, and the features of the correlated sequences are combined to re-detect frames with low-level cognition. Subsequently, the results before and after re-detection are dynamically updated based on fuzzy logic, improving action detection performance. Specifically, this study proposes a fuzzy cognitive update strategy that includes three main parts: constructing correlated sequences, re-detecting frames with low-level cognition, and updating frames with low-level cognition.The construction of correlated sequences. Frames with highlevel cognition are used to construct correlated sequences that are applied in subsequent re-detection processes. As action detection continues, the size of the correlated sequences gradually increases. The matching of the frame with low-level cognition and all the frames with high-level cognition significantly affected the detection speed. Therefore, for a specific frame with low-level cognition 𝐼 𝑖 , only video frames within a certain neighboring range are employed to construct the correlated sequence 𝑇 𝑖 .EQUATION)where 𝜆 is a hyperparameter used to determine the number of correlated sequences around frames with low-level cognition.Re-detection of frames with low-level cognition. The correlated sequences provide rich contextual clues to determine the category of the current frame. It also contains noise that affects the determination of subsequent frame categories. Therefore, the core idea of re-detection is to simulate local evolution by dynamically aggregating the features of frames with low-level cognition and their related sequences. Specifically, we extract features based on the backbone network in [27] . For the extracted feature 𝑓 𝑗 of the j-th frame image in the correlated sequence 𝑇 𝑖 , 𝑓 𝑗 is converted into the key and value spaces, where the former is responsible for comparing similarities, and the latter can be utilized for feature aggregation. It is expressed as:EQUATIONwhere Φ 𝑘 and Φ 𝑣 denote the key mapping convolutional layer and value mapping convolutional layer in the re-detection network, respectively. Then, the cosine similarity is calculated to measure the similarity between the feature of the correlated sequence 𝑓 𝑘 𝑗 and the feature of the frame with low-level cognition 𝑓 𝑘 𝑖 , as follows:EQUATIONwhere 𝜀 𝑗,𝑖 refers to the similarity between the j-th frame of the correlated sequence 𝑓 𝑘 𝑗 and 𝑓 𝑘 𝑖 . The similarity set 𝜀 1,𝑖 , • • • , 𝜀 𝑗,𝑖 is obtained from above, 𝑗 ∈ (1, 2, • • • , 𝑧) is the number of frames in the correlated sequence, and softmax is employed to normalize and obtain the attention mask { ε}. The feature values in the correlated sequence are aggregated to obtain features with high-level cognition f 𝑖 :EQUATIONThen, the features of the frames with low-level cognition 𝑓 𝑣 𝑖 and high-level cognition f 𝑖 are used to obtain the predicted classification score 𝐻 𝑖 after re-detection. EQUATIONEQUATION)Update frames with low-level cognition. From the above steps, confidence Ĉ𝑖 after re-detection is obtained. However, updating the results is unreliable when only confidence is used after re-detection. Therefore, it is necessary to replace 𝐶 𝑖 with Ĉ𝑖 as the input for fuzzy inference and use Equations ( 5)-( 8) to obtain the effectiveness û𝑖 after the re-detection. The result is dynamically updated based on the effectiveness before and after re-detection.EQUATIONwhere 𝑢 𝑖 is the effectiveness of the initial frame and û𝑖 refers to the effectiveness of the frame after re-detection. 𝜏 denotes a predefined threshold. Because the initial frame is the foundation of the detection process, the result's update occurs only when the combined effectiveness of the initial frame and the predefined threshold is lower than the effectiveness obtained after re-detection.Figure 6 shows the overall process of the FCS. First, the correlated sequences of frames with high-level cognition are constructed for further detection. The similarity between the features in the correlated sequence and frames with low-level cognition is then measured, and the features are fused to obtain features with highlevel cognition. The features of frames with low-level cognition and correlated sequences are jointly considered for re-detection. Fuzzy logic is then employed to dynamically update the detection results with low-level cognition before and after re-detection. 
experimental results and analysis: 
 To evaluate the superiority of Cefdet for action detection, experiments are conducted using JHMDB [28] and UCF101-24 [29] . The following introduces the JHMDB and UCF101-24 datasets and describes their implementation details. The experimental results of Cefdet are then quantitatively and qualitatively discussed to demonstrate its effectiveness.JHMDB [28] is a benchmark dataset for action detection. It consists of a video composed of 928 temporary clips from 21 different action categories. The dataset involves fine-grained actions and subtle temporal cues, requiring precise temporal localization for accurate detection. UCF101-24 [29] is derived from the UCF101 dataset and focuses on 24 action categories consisting of 3207 videos representing various human activities such as walking, jogging, basketball, and dancing. The videos exhibited variations in viewpoint, scale, background clutter, and lighting conditions, posing challenges for accurate action detection.Object detector: The experiment extracted keyframes from each video in the JHMDB and UCF101-24 datasets and used the character bounding boxes detected in [30] for inference. The target detector adopted a Faster RCNN [31] in the ResNet-50-FPN backbone. The model is pretrained on ImageNet [32] and fine-tuned on MSCOCO [33] . Backbone: SlowFast [14] is the video backbone. The experiment is instantiated using SlowFast and ResNet-50 pretrained on Kinet-ics700 [34] .Training and evaluation: The model is trained for 7k iterations on the JHMDB dataset, where the first 700 iterations are for linear preheating. SGD is used as the optimizer with a batch size of 8. Similarly, there are 50k iterations of training on the UCF101-24 dataset, with linear preheating applied in the first 1k iterations. The learning rate is 0.0002, which is 10 times less in iterations of 25k and 35k. The hyperparameters 𝜇 1 , 𝜇 2 , and 𝜇 3 are 0.6, 0.2, and 0.2, respectively.To verify the effectiveness of Cefdet in action detection, we compared it with state-of-the-art (SOTA) methods using the JHMDB and UCF101-24 datasets. The frame mean Average Precision (mAP) with an intersection over union (IoU) threshold of 0.5 is used as the evaluation metric, and the experimental results are shown in Table 2 and 3 .The mAP of SOTA reached 82.9% and 84.8% for the JHMDB and UCF101-24 datasets, respectively. Although the SOTA algorithm exhibited advanced performance on the JHMDB and UCF101-24 datasets, its detection accuracy for high-similarity actions remained poor. Moreover, it is difficult for a detector to explore the relationship between continuous actions, resulting in detection results that do not conform to human action norms. Repairing these detection results with cognitive abnormalities is a challenge in action detection because of the lack of effective judgment of the detection results.However, Cefdet achieved mAP values of 84.0% and 85.0% on the JHMDB and UCF101-24 datasets, respectively, which provided gains of 1.1% and 0.2%, respectively, compared with SOTA. Cefdet utilizes fuzzy inference to simulate a cognition-based detection [37] V+F 76.7 85.9 84.0 ACRN(2018) [17] V+F 77.9 -80.0 CA-RCNN(2020) [38] V 79.2 --WOO(2021) [39] V 80.5 --TubeR(2022) [40] V+F -87.4 82.3 SE-STAD(2023) [41] V 82.5 --MCA-SVMM(2023) [42] V 74.9 82.1 81.8 HIT(2023) [20] V 4 indicate that the detection accuracy of action categories such as "shoot_ball", "walk", and "stand" is poor on the JHMDB dataset. This indicates that it is difficult to distinguish between similar actions using existing methods. However, Cefdet exhibited excellent performance in these action categories, with improvements of 12.2%, 10.5%, and 7.2%, respectively, compared with the SOTA algorithm. This improvement is attributed to evaluating the effectiveness of the fuzzy inference system. It simulates a cognition-based detection process and accurately identifies the location of the detection results with cognitive abnormalities. Subsequently, the detection results are divided into frames with high and low-level cognition based on their effectiveness. Frames with low-level cognition are re-detected by similarity-weighted fusion of local features, and frames with higher-level cognition select the results before and after re-detection. This effectively repairs detection results with cognitive abnormalities.The above experimental results demonstrate that Cefdet achieves SOTA performance on the JHMDB and UCF101-24 datasets. This fully reflects Cefdet's superiority and robustness.In this section, ablation experiments are conducted on JHMDB to investigate the effectiveness of FCM and FCS. The frame mAP with an IoU threshold of 0.5 is used as an evaluation metric, and the experimental results are shown in Table 5 and 6 .Notably, when FCM is removed, the confidence score for action detection is selected as the evaluation criterion. The obtained mAP is 83.2%, which is a reduction of 0.8%, indicating that effectiveness had a more significant effect than confidence. Table 6 presents the results of using confidence and effectiveness as the evaluation criteria. The effectiveness results as the evaluation criterion are consistently higher than those of using confidence under different thresholds, and the highest mAP of 83.97 is achieved at a threshold of 0.35, demonstrating the superiority of FCM and indicating The experimental results are updated based on confidence before and after re-detection after removing the FCS. The result is a 0.2% decrease, demonstrating that using only confidence scores to update the experimental results is unreasonable. Cognition-based detection exhibits excellent performance in handling nonlinear, time-varying, and unclear information, effectively improving the accuracy of detecting cognitive abnormalities.The results demonstrated that both the FCM and FCS effectively enhanced the accuracy of action detection in complex scenarios.Figure 7 illustrates the visual results of Cefdet on the JHMDB and UCF101-24 datasets, which reflects the superiority of Cefdet in repairing the detection results with cognitive abnormalities. This improvement is attributed to the combination of human action features and fuzzy inference to simulate a cognition-based detection process. It accurately localizes the detection results of cognitive abnormalities. Furthermore, fuzzy logic is utilized to aggregate the local features of frames with low-level cognition for re-detection, and the detection results are dynamically updated based on the effectiveness of the frames before and after re-detection. This effectively repairs the detection results with cognitive abnormalities.These results indicate that Cefdet performs superiorly in these challenging situations and is more suitable for action detection. 
conclusion and future work: 
 This study proposed Cefdet, which introduced the concept of "cognition-based detection" into action detection to simulate human cognition. First, the FCM is designed to evaluate the effectiveness of the frames. It extracts various action features from video sequences and inputs them into fuzzy inference. The cognition-based detection process is simulated using a fuzzy system to obtain effectiveness, which accurately locates the position of action frames with cognitive abnormalities. Then, the FCS is proposed based on the FCM to update the results dynamically. It constructs related sequences by frames with high-level cognition and performs a weighted fusion of features by fuzzy inference for re-detection. Subsequently, the results before and after re-detection are dynamically updated, which effectively repairs the detection results with cognitive abnormalities. Experiments on public datasets proved that Cefdet achieves superior performance and promotes the further application of fuzzy inference in action detection.In future work, our objective will be to incorporate interval type-2 fuzzy set theory into developing a robust and real-time action detection framework and further facilitate the application and advancement of fuzzy theory in action detection.  
 
 Appendix:  


Decision: All reviewers are satisfied with the responses and contributions made by the submission. The authors are suggested to take all the comments into consideration for final version.

Reviews: review: paper_summary: paper_summary: The paper introduces a cognitive effectiveness network based on fuzzy inference (Cefdet) that aims to enhance the accuracy and logical consistency of action detection results by simulating human cognitive processes. To address the shortcomings of existing action detection methods that often overlook the evaluation of detection effectiveness and produce abnormal cognitive detection results, the study incorporates the concept of "cognition-based detection." The contributions of this work is the establishment of a Fuzzy-Driven Cognitive Effectiveness Evaluation Module (FCM) and a Fuzzy Cognitive Update Strategy (FCS).Combining these two modules can verify the validity of the detection results and repair abnormal cognitive detection results.
strength_weakness: strength_weakness: FCM comprehensively considers the confidence of each frame, the correlation between adjacent frames and the position score, uses the fuzzy inference system to accurately locate the position of the action frame with cognitive abnormalities.FCS  divides the detection results into high and low levels according to FCM, and 
redetects the abnormal detection results through weight fusion of local features. FCS then uses fuzzy logic to dynamically update the results with low-level cognition before and after re-detection.
Through fuzzy logic , Cefdet achieves accurate evaluation and dynamic optimization of action detection results, solves the shortcomings of existing methods in judging and detection effectiveness, and effectively improves the accuracy and reliability of action detection.Experimental results on public datasets demonstrate Cefdet's superiority over several mainstream algorithms , which verifies its effectiveness and advantages.
limitations: limitations: While the Cefdet methodology significantly improves action detection accuracy, it is not without limitations.
1.The formulation of fuzzy inference rules is subjective, and the principle of rule setting and the process of rule optimization are not described in detail.
2.The performance of both the FCM and FCS modules is highly dependent on the quality of input features.Try to verify the effectiveness of the method in the case of some noise.
3.Although the paper highlights the effectiveness of Cefdet, it does not extensively discuss computational efficiency and resource consumption.
paper_summary: paper_summary: The paper proposes a cognitive effectiveness network for action detection based on fuzzy inference (Cefdet), which introduces the concept of "cognition-based detection" to simulate human cognition. Cefdet includes a fuzzy-driven cognitive effectiveness evaluation module (FCM), which integrates fuzzy inference into action detection and combines it with human action features to simulate the cognition-based detection process. A fuzzy cognitive update strategy (FCS) based on FCM is proposed to effectively update the results with cognitive abnormalities.
strength_weakness: strength_weakness: The proposed cognitive effectiveness network (Cefdet) for action detection based on fuzzy inference demonstrates several strengths as follows:
1)	Cefdet introduces the concept of "cognition-based detection" to address the limitations of existing methods in action detection. By simulating human cognition, it offers a fresh perspective on improving detection effectiveness.
2)	The paper appropriately applies fuzzy logic and fuzzy inference techniques to action detection, providing a solid technical foundation for the proposed Cefdet network. The integration of human action features further strengthens the technical correctness of the approach.
3)	The paper introduces fuzzy inference into action detection, incorporating rich input information. It takes human action features, including confidence per frame, correlation between adjacent frames, and position scores per frame, as inputs and integrates them into the fuzzy inference system.
limitations: limitations: 3.	Limitations
1)	Fuzzy reasoning itself may have limitations. In the ablation experiments, it seems that the influence of threshold settings on confidence and effectiveness is limited. The portions determined through fuzzy reasoning without further processing are not validated for correctness, which could potentially impact the final action detection results. Therefore, considering the addition of validation for non-abnormal results may be necessary.
2)	In FCS, the fusion processing of low-level cognitive abilities and high-level cognitive abilities is currently limited to aggregation. However, there is room for improvement in the fusion processing approach.
3)	Simulating human cognition should be a relatively complex issue, considering only confidence makes it difficult to accurately simulate adjacent frames. Further research and exploration are needed to consider human cognition, which is also a black box problem. 
4)	Adding an additional dataset to the experiment seems more convincing
paper_summary: paper_summary: The manuscript proposes a cognitive validity network (Cefdet) based on fuzzy reasoning and introduces the concept of "cognition-based detection" to simulate human cognition. First, a fuzzy-driven cognitive validity assessment module (FCM) is proposed to introduce fuzzy reasoning into action detection. Then, a fuzzy cognitive update strategy (FCS) is proposed based on FCM, which uses fuzzy logic to re-detect cognitive-based detection results and effectively update cognitive abnormal results.
strength_weakness: strength_weakness: 1.The manuscript introduces the cognitive detection process and fuzzy inference process into the action detection task, which is very interesting and innovative.
2.The method proposed in the manuscript achieves advanced results on two data sets.
limitations: limitations: 1.The JHMDB or UCF101-24 used in the manuscript experiments are two small datasets in the action detection task, and it is curious to see how the effectiveness of the method proposed in the manuscript performs on datasets of large size or many categories.
2.I'm curious about the connection between fuzzy rules and actions. The fuzzy inference process seems to have many rules. How to reason about actions based on the rules?
3.The manuscript does not mention the loss function optimization process of the method.
4.How is the Updater in Figure 2 implemented? This is not explicitly done in the method.
5.The manuscript focuses action detection, and the methods are updated by detecting abnormal actions. Some work related to these is missing, such as:
[1]Hierarchical scene normality-binding modeling for anomaly detection in surveillance videos[C]//Proceedings of the 30th ACM international conference on multimedia. 2022: 6103-6112.  

[2] Egocentric Early Action Prediction via Multimodal Transformer-Based Dual Action Prediction. IEEE Trans. Circuits Syst. Video Technol. 33(9): 4472-4483 (2023)

[3] A Knowledge-Based Hierarchical Causal Inference Network for Video Action Recognition[J]. IEEE Transactions on Multimedia, 2024. 

[4] Eval: Explainable video anomaly localization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.