# A Survey on Video Big Data Analytics: Architecture, Technologies, and Open Research Challenges
## 3 VBDA Architectures and Core Technologies
### VBDA Core Technologies
#### 3.2.1 Scalable Big Data Processing Frameworks  
The transition from batch-oriented to real-time stream processing has fundamentally reshaped the architecture and responsiveness of VBDA systems. As summarized in Table 3, Apache Kafka has established itself as a cornerstone for real-time ingestion and message queuing, offering high throughput, scalability, and robust fault tolerance through its distributed architecture [110]. Apache Spark remains a widely adopted engine, valued for its ability to unify batch and micro-batch processing, thus supporting scalable analytics across both historical and near-real-time video streams [10].  
\begin{table}
\begin{tabular}{l c c c c c c c c c c c} \hline \hline
**Reference** & **Year** &
\begin{tabular}{c} **Hadoop** \\ **(Map Reduce)** \\ \end{tabular} & **Kafka** & **Spark** & **Flink** & **NiFi** & **Hive** & **Impala** & **Pig** & **Storm** \\ \hline [PERSON] et al. [108] & 2024 & & & ✓ & & & ✓ & & & & \\ [PERSON] et al. [111] & 2023 & & ✓ & & ✓ & & ✓ & & & & \\ [PERSON] et al. [10] & 2023 & & ✓ & ✓ & ✓ & & ✓ & & & & \\ [PERSON] et al. [110] & 2023 & & ✓ & & ✓ & & ✓ & & & & \\ [PERSON] et al. [112] & 2022 & & ✓ & & ✓ & & & & & & \\ [PERSON] et al. [113] & 2022 & & ✓ & & ✓ & & & ✓ & & & \\ [PERSON] et al. [114] & 2021 & & & ✓ & & & & ✓ & ✓ & & \\ [PERSON] et al. [115] & 2021 & & ✓ & ✓ & & & & & ✓ & & ✓ \\ [PERSON] et al. [116] & 2021 & & & ✓ & ✓ & & & ✓ & ✓ & & \\ [PERSON] et al. [21] & 2020 & ✓ & ✓ & ✓ & & & & & & & \\ [PERSON] et al. [107] & 2020 & & ✓ & ✓ & ✓ & & & & & & \\ [PERSON] et al. [11] & 2020 & ✓ & & ✓ & ✓ & & & & & & \\ [PERSON] et al. [117] & 2020 & ✓ & & ✓ & ✓ & & & & & & \\ [PERSON] et al. [118] & 2020 & & & ✓ & ✓ & & & & & & \\ [PERSON] et al. [119] & 2019 & & ✓ & ✓ & ✓ & & & & & & \\ [PERSON] et al. [120] & 2019 & ✓ & ✓ & ✓ & & & & ✓ & & ✓ \\ \hline \hline \end{tabular}
\end{table}
Table 3: Scalable big data processing frameworks for VBDA.  
Apache Flink has gained traction for its native support of true stream processing and stateful computations, enabling low-latency analytics in applications such as anomaly detection, traffic monitoring, and intelligent surveillance [11]. Complementing these systems, Apache NiFi is increasingly used to orchestrate end-to-end VBDA pipelines, offering low-code automation for ingesting, transforming, and routing video data between edge and cloud components [113].  
Although Hadoop MapReduce previously played a central role in batch video analytics, its relevance has declined in modern VBDA pipelines due to inherent latency and a lack of flexibility. It is now primarily used for back-end archival processing and long-term video storage, rather than front-line real-time inference [30; 107].  
More recently, Apache Pulsar ([[https://pulsar.apache.org](https://pulsar.apache.org)]([https://pulsar.apache.org](https://pulsar.apache.org)), accessed on 18 May 2025) has emerged as a potential alternative or complement to Kafka. Its support for multi-tenancy, geo-replication, and a decoupled architecture for storage and computing make it suitable for hybrid cloud-edge deployments. Although its adoption in VBDA remains limited as of 2024, its architectural flexibility positions it as a promising component in next-generation, AI-integrated video analytics systems [10].