# AUD-SUR: An Audio Analyzer Assistant for Audio Surveillance Applications

Lam Pham, Phat Lam, Dat Tran, Alexander Schindler, Silvia Poletti, Marcel

Hasenbalg, David Fischinger, Martin Boyer

► To cite this version:

Lam Pham, Phat Lam, Dat Tran, Alexander Schindler, Silvia Poletti, et al. AUD-SUR: An Audio Analyzer Assistant for Audio Surveillance Applications. Proceedings of the 19th Linux Audio Conference, Jun 2025, Lyon, France. hal-05096011

HAL Id: hal-05096011

https://hal.science/hal-05096011v1

Submitted on 3 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# AUD-SUR: AN AUDIO ANALYZER ASSISTANT FOR AUDIO SURVEILLANCE APPLICATIONS

Lam Pham¹*, Phat Lam²*, Dat Tran³, Alexander Schindler¹, Silvia Poletti¹, Marcel Hasenbalg¹, David Fischinger¹, Martin Boyer¹

¹ Austrian Institute of Technology, Vienna, Austria
{lam.pham, alexander.schindler, silvia-poletti, hasenbalg.marcel, fischinger.david, martin.boyer}@ait.ac.at

² Ho Chi Minh City University of Technology, Ho Chi Minh City, Vietnam

phat.lamhcmutddk21@hcmut.edu.vn

³ FPT University, Ho Chi Minh City, Vietnam

datttse170375@fpt.edu.vn

## ABSTRACT

In this paper, we present an audio analyzer assistant tool designed for a wide range of audio-based surveillance applications (This work is a part of our defame Fakes and EUCINF projects). The proposed tool, referred to as Aud-Sur, comprises two main phases: Audio Analysis and Audio Retrieval, respectively. In the first phase, multiple open-source audio models are leveraged to extract information from input audio recording uploaded by a user. In the second phase, the user interacts with the Aud-Sur tool via a natural language question-and-answer manner, powered by a large language model (LLM), to retrieve the information extracted from the processed audio file. The Aud-Sur tool was deployed using Docker on a microservices-based architecture design. By leveraging open-source audio models for information extraction, LLM for audio information retrieval, and a microservices-based deployment approach, the proposed Aud-Sur tool offers a highly extensible and adaptable framework that can integrate more audio tasks, and be widely shared within the audio community for further development.

*Items*— audio, surveillance, pre-trained model, large language model, retrieval.

## 1. INTRODUCTION

With advancements in neural network architectures and deep learning techniques, various audio-based analysis tasks have been proposed by the research community such as Audio Captioning, Audio Event Detection, Speech-to-Text, Speech Emotion Detection, etc. However, these tasks are often proposed for specific purposes and evaluated on certain benchmark datasets. For instance, audio captioning, audio event detection, speech to text, speech emotion have been evaluated using the large benchmark datasets such as Clotho [1], Audioset [2], Librispeech [3], and IMOCAP [4], respectively. Recently, pretrained audio models leveraging self-supervised learning strategies have been proposed, demonstrating potential across different audio tasks [5, 6, 7]. However, these models are designed and verified on certain relevant audio tasks (i.e., mainly human speech analysis), whereas a wide range of audio processing tasks requires different audio engineering features. For example, while Mel filter banks are suitable for Speech-To-Text task, linear filters are more effective for Deepfake Speech Detection [8]. Notably, in Acoustic Scene Classification (ASC), a combination of multiple spectrograms of Gammatonegram (GAM), Constant-Q Transform (CQT), and Mel-spectrogram (MEL) has been shown to enhance model performance [9]. Furthermore, these pre-trained audio models present the high computational complexity and require the fine-tuning process for downstream audio tasks.

In real-life audio-based applications, multiple audio tasks have to be integrated into complex systems rather than relying on a standalone task. For example, a call center support tool

<figure><img src="image_2.png" /><figcaption>Figure 1: The high-level architecture of proposed Aud-Sur tool</figcaption></figure>

may need to integrate Speech Denoise, Eco Reduction, Speech-To-Text, Speaker Identification, etc. Therefore, the ability to seamlessly integrate new audio tasks is essential for real-world audio-based applications. In the context of audio surveillance, these applications focus on audio event detection and audio tracking [10]. Existing audio surveillance systems are often designed for specific tasks such as anomaly violence detection [11], detection of children abuse [12], or riot context detection [9, 13]. However, this task-specific approach limits the ability to integrate or extend new audio tasks in the future. Furthermore, publications on audio surveillance often focus on model development rather than proposing a deployment process, resulting in a lack of comprehensive solutions [14]. To address the challenge of deploying a comprehensive solution for audio surveillance that not only focuses on model development but also provides a user-friendly practical usage, we propose:

* We leverage state-of-the-art open-source audio models, each excel in a specific task. This approach allows seamless upgrades, where an outdated model can be replaced with a more advanced one without requiring major system modifications.

* Unlike conventional audio surveillance systems that focus only on Audio Event/Scene Detection, our tool integrates a broader range of tasks such as Speech-to-Text, Speech Emotion Detection, Audio Captioning, Language Detection to provide richer context and deeper situational awareness.

* We use a large language model (LLM) to enable natural and human-like interaction between our proposed system and the user. This enhances accessibility and makes the tool more user-friendly for both technical and non-technical users.

* To enhance scalability and adaptability, we design the proposed system using a microservices-based architecture and Docker technique. This approach allows seamless updates,

<footnote>* Main and equal contribution into the paper.</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 2: Logical processing flow in the Audio Analysis Phase (AAP)</figcaption></figure>

parallel execution, and easy extending and integration of new models in the future.

## 2. HIGH-LEVEL ARCHITECTURE

Figure 1 presents the high-level architecture of the proposed Audio tool. In overall, the tool comprises three main components. The first component is the User Interface (UI). The UI serves as an entry point where users interact with the tool by uploading an audio file and submitting a query or question, typically in natural language, to request specific information about the audio. The second component is the Audio Analysis Phase (AAP) which processes the uploaded audio recording by performing various audio tasks. The AAP component extracts task-structured results representing multiple aspects of the audio input. For each audio task, the results are exported in a JSON file, which is then aggregated and standardized to ensure consistency across factors such as time boundaries and missing values in the JSON results from other audio tasks. Finally, the Audio Retrieval Phase (ARP) utilizes the standardized results to interpret the extracted information and retrieve relevant insights based on the user's question, formulating a corresponding response in natural language.

### 2.1. Audio Analysis Phase (AAP)

The Audio Analysis Phase (AAP) aims to extract structured information representing different aspects of input audio recording. The extracted data serves as the main reference resource for generating responses in the consequent Audio Retrieval Phase (ARP). Since the quality of the extracted information directly impacts the accuracy and relevance of the final response, ensuring high-quality results in this phase is crucial. Therefore, instead of relying on a single model to perform multiple tasks, we employ multiple individual models, each of which was optimized for a specific audio task and achieved high performance on benchmark datasets.

To describe the APP in detail, we present Figure 2, which provides a comprehensive flowchart for processing input audio. In particular, at the top of the flow chart (Figure 2A), the APP process begins with the Preprocessing module, which performs multiple steps such as resampling, filtering, and changing representation format (e.g., raw audio, spectrogram, MFCC) to ensure

<table><caption>Table 1: Individual audio models used in the AAP component</caption><thead><tr><td>Models</td><td>Tasks</td><td>Licenses</td></tr></thead><tbody><tr><td>Whisper [5]</td><td>Speech to Text (S2T)<br/>Language Detection (LD)</td><td>MIT</td></tr><tr><td>mBart [15]</td><td>English Translation (ET)</td><td>MIT</td></tr><tr><td>PANN [16]</td><td>Acoustic Event Detection (AED)</td><td>MIT</td></tr><tr><td>pyannote [17]</td><td>Speaker Count (SC)</td><td>MIT</td></tr><tr><td>ECAPA-TDNN [18]</td><td>Voice Gender Detection (VGD)</td><td>MIT</td></tr><tr><td>SpeechBrain [19]</td><td>Speaker Diarization (SD)</td><td>Apache license 2.0</td></tr><tr><td>Wav2vec2.0 [19, 6]</td><td>Speech Emotion Recognition (SER)</td><td>Apache license 2.0</td></tr><tr><td>KD [20]</td><td>Audio Captioning (AC)</td><td>Apache license 2.0</td></tr><tr><td>AIT-ASC [9, 21]</td><td>Acoustic Scene Classification (ASC)</td><td><strong>Our development</strong></td></tr><tr><td>AIT-DSD [22, 23, 8]</td><td>Deepfake Speech Detection (DSD)</td><td><strong>Our development</strong></td></tr></tbody></table>

the alignment between the input audio and each of pre-defined single models. Then the preprocessed audio is fed into multiple audio models to extract audio information. To assure a more organized and effective extraction of relevant information from an audio recording, we categorize multiple audio models in this phase into three main groups, referred to as the sound-based tasks (Figure 2B), speech-based tasks (Figure 2C), and summary-based tasks (Figure 2D). These three task groups analyze the input in parallel. Notably, for speech-based tasks, a preliminary speaker count module [17] is first leveraged to detect the presence of human speech. If at least one speaker is identified, the system continues speech-based tasks processing; otherwise, these tasks are skipped to optimize computational efficiency.

For each task group, the detailed process is presented in Figure 2B, Figure 2C, and Figure 2D, respectively. The first group of sound-based tasks focuses on analyzing the background environment and non-speech elements within the recording. They include specific audio tasks of Acoustic Scene Classification (ASC) [9, 21] (e.g., recordings into one of multiple pre-defined acoustic scene classes such as 'airport', 'park', 'outdoor', 'in door', etc.), Acoustic Event Detection (AED) [16] (e.g., recognizes specific auditory events such as 'sirens', 'alarms', or 'footsteps', etc). As the tool is applied for surveillance applications, we highly focus on detecting events indicating violent contexts such as 'gun', 'fireworks', 'screaming', 'explosion', etc, by adding violent/alarm tags on the task results for the next ARP processing. As Figure 2B describes, we employ sound-based tasks using 10-second segments instead of input recording for greater flexibility in handling variations in background environment.
<figure><img src="image_4.png" /><figcaption>Figure 3: Zero-shot Prompting with Filtering & Response control (ZPFR)</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4: Chain-of-thought (CoT) prompting</figcaption></figure>

ments and sound events. Then, the segment-based results of these two models are gathered and collectively referred to as 'Acoustic information'. Notably, before finalizing the AED results, we apply an additional refinement step, in which we assign a 'violent' tag to any segments containing at least one event indicative of violent behaviors. Among the 500+ class labels in AudioSet [2], which the AED model was trained on, we manually selected a subset of class events associated with violence or riots and used them as label references for this tagging process. This serves as additional auditory cues in the next ARP to better enhance situational awareness for the main purpose of audio surveillance.

The second group, referred to as speech-based tasks, is designed to extract information related to human speech. These incorporate Speech-To-Text (S2T) [5] (e.g., transcribing spoken content into text), Speaker Diarization (SD) [19] (e.g., segmenting audio based on personal identity or simply identifies 'who spoke when'), Speech Emotion Recognition [19, 6] (e.g., determining the speaker's emotional state) and Language Detection (LD) [5] (e.g., detecting the spoken language). The flow to process results in this task group is described in Figure 2C. First, a Speech-To-Text model (e.g., Whisper) is utilized to generate text transcriptions, which are segmented based on their respective time boundaries. Since Whisper is a general-purpose model, we derive language identification results directly from the language tag in its multitask sequence tokens. Using the time boundaries provided by Speech-To-Text model, we further divide the audio input into segments, which then serve as inputs for the three following models: Speech Emotion Recognition (SER), Speaker Diarization (SD), and Voice Gender Detection (VGD). In other words, these models process audio segments defined by Whisper's segmentation. Assuming Whisper's time boundary estimation is sufficiently accurate, this approach of segmentation ensures that speaker-related attributes such as identity, gender, and emotion are captured precisely at the moment of speech, reducing errors associated with multiple speakers overlapping in a single segment when using fixed-length segments that could affect the performances of the next three single models. The results of single models in the speech-based tasks are also aggregated, referred to as 'human speech information'.

The final group is referred to as the summary-based tasks which provides a high-level understanding of the audio content. The tasks include Audio Captioning (AC) [20], which generates textual descriptions summarizing the content of an audio recording, and Deepfake Speech Detection (DSD) [22, 23], which determines whether a given speech sample has been artificially manipulated or is authentic. As illustrated in Figure 2D, models in this category process input audio based on fixed-length segments. The Audio Captioning (AC) model operates on 10-second segments to accommodate potential shifts in the overall content of the recording, while the DSD model processes 4-second segments, which are sufficient to capture distinctive artifacts in

human speech. The final output of the DSD model is obtained through majority voting on the segment-wise results. Again, the outputs from this task group are then gathered and collectively referred to as 'other information'.

Given JSON file results from task groups, these JSON files are then aggregated and standardized. The Standardize module aims to re-structure the final information of the audio recordings. In our work, we employ a timeline-centric standardization approach that arranges the results based on a shared timeline, where single task's outputs are aligned according to their occurrence throughout the audio recording. By structuring the AAP into such content-based categories, our proposed tool ensures a systematic and scalable approach to extracting meaningful insights from recordings. This modular framework allows for greater expansion and management (i.e., new tasks or improved models can be categorized and integrated without re-designing the entire system).

## 2.2. Audio Retrieval Phase (ARP)

The Audio Retrieval Phase (ARP) plays a crucial role in bridging the gap between audio analysis results from the AAP and a human-interpretable response. Given the final information JSON output from the AAP component, which may be fragmented or lack context, the ARP leverages a Large Language Model (LLM) to process and contextualize the information and then transfer it into a coherent and informative response tailored to the user's query. Since the application is designed for surveillance purposes, the ARP needs to address several challenges including irrelevant queries/questions (e.g., A user asks about topics unrelated to the audio content, leading to off-topic or unhelpful responses), LLM Hallucinations (e.g., the LLM might generate inaccurate or misleading information despite being provided the extract audio content), etc. To ensure reliable and contextually relevant responses, we explore two prompting techniques:

**Zero-shot Prompting with Filtering & Response Control (ZPFR):** Zero-shot prompting is a technique in which an AI model is given a task or question without any prior examples or specific training, relying solely on its pre-existing knowledge and prompt to generate a response [24]. While effective in many cases, zero-shot prompting alone may not guarantee precision or adherence to specific guidelines. To improve response quality, we integrate filtering rules into the system prompt to pre-process user queries and eliminate irrelevant inputs. Additionally, response control rules are applied to guide the model in generating outputs that align with pre-defined objectives. An example of such type of prompting is illustrated in Figure 3.

**Chain-of-Thought Prompting:** Chain-of-thought (CoT) prompting [25] is a technique that significantly improves the reasoning capabilities of large language models (LLMs). Instead of simply asking for a final answer, CoT prompting encourages the model to break down complex problems into a series of intermediate steps, mimicking human-like reasoning. Based on this
<figure><img src="image_6.png" /><figcaption>Figure 5: Deployment of our proposed AUD-SUR tool as multiple services leveraging Docker</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 6: The user interface (UI) of our proposed Aud-Sur tool</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Figure 7: Our proposed Aud-Sur tool operates as an interactive assistant</figcaption></figure>

mechanism, we try to guide the LLM toward various reasoning steps to implicitly handling irrelevant queries and reducing hallucination above before generating final responses. An example of this type of prompt can be described as Figure 4.

## 3. DEPLOYMENT

### 3.1. Module Implementation

The deployment of our proposed Aud-Sur tool comprehends the design of a User Interface (UI), independent audio models used in the AAP component, and LLM in the ARP component. First, User Interface (UI) is implemented by Streamlit platform [26]. Regarding audio models in the AAP component, the Table 1 presents the pre-trained audio models along with their respective licenses for specific audio tasks. To showcase the feasibility of expanding the scope of the proposed AAP and share further contributions in the context of audio surveillance with the research community, we integrate the APP with our self-deployed models of Acoustic Scene classification and Deepfake Speech Detection, along with open-source models for other audio tasks. All audio models utilize deep neural networks and are implemented using the PyTorch framework. Regarding the ARP component, we use two approaches to leverage the operation of Large Language Model (LLM). The first approach involves interacting with the

LLM via an API, while in the second one, we run the LLM locally, providing greater control and data privacy. In both cases, we utilize LLaMA [27] as our LLM with the following configurations: a temperature of 0.3, a maximum token limit of 256 to constrain responses, and a top-p value of 0.5 for nucleus sampling.

### 3.2. Deployment Environment

We utilize Docker and FastAPI to deploy our proposed Aud-Sur tool following a client-server architecture as illustrated in Figure 5. The client side consists of a user interface (UI). This UI allows users to interact with the system and submit queries. The queries and response results are processed with the support from prompting designs and a Large Language Model (LLM). The server side is structured as multiple independent services, with each service responsible for a predefined group of tasks, as mentioned in Section 2. In other words, each service includes some audio components which use the same Python package libraries and Conda dependencies. Each service also includes an independent API which is used to communicate via predefined ports and collectively generate results for the client side.

Regarding Docker containerization, described at the bottom of Figure 5, we first construct a base Docker image that includes common dependencies (e.g., Conda environments, GPU-enabled
<figure><img src="image_9.png" /><figcaption>Figure 8: Compare LLM response using ZPFR prompting and CoT prompting. The ‘record_1.wav’ involves a confidential reconnaissance unit report between two German speakers.</figcaption></figure>

inference installations, and essential libraries). Building upon
this base image, each service is packaged into a separate Docker
image and run independently in its own Docker container. The
services interact via API calls and results are transferred to the
client through port connections. By following this approach, we
achieve a modular, scalable and efficient deployment process,
ensuring independent execution of all system functionalities.

4. DEMONSTRATION AND DISCUSSION

This section illustrates a live demonstration of our proposed Aud-
Sur tool. Figure 6 presents the User Interface, where users can
upload an audio file in `.wav` or `.mp3` format and submit text-
based queries or questions to inquire information related to the
uploaded audio. The tool processes each query/question by ana-
lyzing the input, extracting relevant information, and then gen-
erating a corresponding response. The interaction between the
user and the system is depicted in Figure 7. Unlike conventional
methods that rely on a single model’s input-output pipeline, the
integration of a LLM in the ARP component of our proposed
Aud-Sur tool enables the tool function in a chatbot-like manner
that facilitates a more dynamic and context-aware approach to
information retrieval. This approach overcomes the limitations
of standalone audio models, particularly in cases where a user’s
query requires information from multiple models or consists of
multiple sub-questions. For instance, in Figure 7, the user’s third
query "What are the speakers in this recording talking about?
How do they feel when talking to each other?" requires results
from both Speech-to-Text and Speech Emotion Detection. Lever-
aging multiple-model approach combined with the support of
LLM, our Aud-Sur tool could combines information from multi-
ple sources to generate a comprehensive response. Additionally,
with prompting support, our system can filter out queries that
are irrelevant to the audio content. As illustrated in Figure 7, the
system politely rejects off-topic questions (highlighted in red)
and instead prompts the user with more contextually appropriate
questions.

To assess the impact of different prompting techniques, we

evaluate our tool’s responses using ZPFR and CoT prompting
on the same set of user queries. Given that our tool is designed
for surveillance applications, we specifically analyze its ability to
monitor and detect audio indicators related to potential threats,
such as extreme audio events, riots, violent sounds, pre-crime
activities, etc. For these purposes, we conduct demonstrations
using two audio samples that contain diverse information rel-
evant to our objectives. Specifically, the first sample, named
‘record 1.wav’, features a confidential reconnaissance unit report
conversation between two German speakers, aiming at evaluat-
ing our system’s ability to extract sensitive intelligence-related
information in human speech. The second one, referred to as
‘record 2.wav’, is created by combining three different audio
recordings with different background scenes, with a riot occur-
ring in the first 15 seconds. The second recording is designed to
test the system’s effectiveness in detecting security threats and
handle context shifting.

As shown in Figure 8 and Figure 9, our Aud-Sur tool successfully extracts relevant information across varied audio environments. The analysis also shows differences in the results using the two prompting techniques. In particular, ZPFR method tends to provide more natural, controlled and compact responses. In contrast, CoT prompting produces more structured and detail-oriented, due to its predefined reasoning steps. These differences highlight the importance of optimizing a prompting strategy to enhance accuracy, reliability, and suitability of LLM-driven real-world audio surveillance applications.

5. CONCLUSION

We have presented Aud-Sur, an audio analyzer assistant tool, for
audio surveillance applications<fnref n="1" /> (i.e., The tool can be directly
used by authorities for early detecting potential crimes). It lever-
ages various audio models for information extraction and inte-
grates a large language model (LLM) for natural language-based
audio information retrieval. With the integration of multiple open-

<footnote n="1">¹The tool was finished and planed for a demo or an access in the LAC conference duration.</footnote>
<figure><img src="image_10.png" /><figcaption>Figure 9: Comparison of LLM responses using ZSFR prompting and CoT prompting. The ‘record 2.wav’ file is created by combining three separate audio recordings, with the first 15 seconds depicting a riot scene.</figcaption></figure>

source audio models and the use of microservices and Docker for deployment, our system enables seamless integration of new audio models/tasks, making the Aud-Sur tool highly adaptable and suitable for a wide range of audio surveillance applications. Inspired by surveillance applications and leveraging open source models, we further analyze and research on video in the future work, in which both audio and image information are explored.

## 6. ACKNOWLEDGMENTS

EUCINF project is co-funded by European Union under grant agreement N°101121418. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Commission. Neither the European Union nor the granting authority can be held responsible for them.

Defame Fakes is funded by the Austrian security research programme KIRAS of the Federal Ministry of Finance (BMF).

<figure><img src="image_11.png" /></figure>

FFG
Forschung wirst

KIRAS
Bibliotheken-forschung

<figure><img src="image_12.png" /></figure>

Bundesministerium
Finanzen

## 7. REFERENCES

[1] Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen, “Clotho: an audio captioning dataset,” in Proc. ICASSP, 2020, pp. 736–740.

[2] Jort F. Gemmeke et al., “Audio set: An ontology and human-labeled dataset for audio events,” in Proc. ICASSP, 2017, pp. 776–780.

[3] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur, “Librispeech: An asr corpus based on public domain audio books,” in Proc. ICASSP, 2015, pp. 5206–5210.

[4] Carlos Busso et al., “Iemocap: Interactive emotional dyadic motion capture database,” Language resources and evaluation, vol. 42, pp. 335–359, 2008.

[5] Alec Radford et al., “Robust speech recognition via large-scale weak supervision,” in Proc. ICML, 2023, pp. 28492–28518.

[6] Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli, “wav2vec 2.0: a framework for self-supervised learning of speech representations,” in Proc. NIPS, 2020.

[7] Sanyuan Chen et al., “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022.

[8] Lam Pham, Phat Lam, Dat Tran, Hieu Tang, Tin Nguyen, Alexander Schindler, Florian Skopik, Alexander Polonsky, and Hai Canh Vu, “A comprehensive survey with critical analysis for deepfake speech detection,” Computer Science Review, vol. 57, pp. 100757, 2025.

[9] Lam Pham, Dat Ngo, Dusan Salovic, Anahid Jalali, Alexander Schindler, Phu X. Nguyen, Khoa Tran, and Hai Canh Vu, “Lightweight deep neural networks for acoustic scene classification and an effective visualization for presenting sound scene contexts,” Applied Acoustics, vol. 211, pp. 109489, 2023.

[10] Marco Crocco, Marco Cristani, Andrea Trucco, and Vittorio Murino, “Audio surveillance: A systematic review,” ACM Comput. Surv., vol. 48, no. 4, 2016.

[11] Jungpil Shin, Abu Saleh Musa Miah, Yuta Kaneko, Najmul Hassan, Hyoun-Sup Lee, and Si-Woong Jang, “Multimodal attention-enhanced feature fusion-based weakly supervised anomaly violence detection,” IEEE Open Journal of the Computer Society, vol. 6, pp. 129–140, 2025.

[12] Jiuqi Yan, Yingxian Chen, and W.W.T. Fok, “Detection of children abuse by voice and audio classification by short-time fourier transform machine learning implemented on nvidia edge gpu device,” in Proc. PRAI, 2023, pp. 893–897.

[13] Lam Pham, Tin Nguyen, Phat Lam, Hieu Tang, and Alexander Schindler, “Toolchain for comprehensive audio/video
analysis using deep learning based multimodal approach: Use case of riot or violent context detection,” in Proc. CBMI, 2024, pp. 1–4.

[14] Dalila Durães, Francisco S Marcondes, Filipe Gonçalves, Joaquim Fonseca, José Machado, and Paulo Novais, “Detection violent behaviors: a survey,” in Ambient Intelligence–Software and Applications: 11th International Symposium on Ambient Intelligence. Springer, 2021, pp. 106–116.

[15] Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer, “Multilingual denoising pre-training for neural machine translation,” *Transactions of the Association for Computational Linguistics*, vol. 8, pp. 726–742, 2020.

[16] Q. Kong, Y. Cao, T. Iqbal, Y. Wang, W. Wang, and M. D. Plumbley, “Panns: Large-scale pretrained audio neural networks for audio pattern recognition,” *IEEE/ACM Transactions on Audio, Speech, and Language Processing*, vol. 28, pp. 2880–2894, 2020.

[17] Hervé Bredin, Ruiqing Yin, Juan Manuel Coria, Gregory Gelly, Pavel Korshunov, Marvin Lavechin, Diego Fustes, Hadrien Titeux, Wassim Bouaziz, and Marie-Philippe Gill, “pyannote.audio: neural building blocks for speaker diarization,” 2019.

[18] Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck, “Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,” in *Proc. INTERSPEECH*, 2020, pp. 3830–3834.

[19] Mirco Ravanelli et al., “SpeechBrain: A general-purpose speech toolkit,” 2021, arXiv:2106.04624.

[20] Xuenan Xu, Haohe Liu, Mengyue Wu, Wenwu Wang, and Mark D Plumbley, “Efficient audio captioning with encoder-level knowledge distillation,” in *Proc. INTERSPEECH*, 2024, pp. 1160–1164.

[21] Lam Pham, Khoa Tran, Dat Ngo, Hieu Tang, Son Phan, and Alexander Schindler, “Wider or deeper neural network architecture for acoustic scene classification with mismatched recording devices,” in *Proc. ACM MM Asia*, 2022, pp. 1–5.

[22] Lam Pham, Dat Tran, Florian Skopik, Alexander Schindler, Silvia Poletti, David Fischinger, and Martin Boyer, “DIN-CTS: Low-complexity depthwise-inception neural network with contrastive training strategy for deepfake speech detection,” *arXiv preprint arXiv:2502.20225*, 2025.

[23] Lam Pham, Phat Lam, Truong Nguyen, Huyen Nguyen, and Alexander Schindler, “Deepfake audio detection using spectrogram-based feature and ensemble of deep learning models,” in *Proc. IS2*, 2024, pp. 1–5.

[24] Yinheng Li, “A practical survey on zero-shot prompt design for in-context learning,” in *Proceedings of the Conference Recent Advances in Natural Language Processing - Large Language Models for Natural Language Processings*, 2023, p. 641–647.

[25] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” in *Proceedings of the 36th International Conference on Neural Information Processing Systems*, 2022.

[26] “Streamlit platform,” https://streamlit.io/, 2021.

[27] Hugo Touvron et al., “Llama: Open and efficient foundation language models,” 2023.