# Engineering Resource-Efficient Data Management for Smart Cities with Apache Kafka +
## 4 Data Corpus-as-a-Service  
In the context of the H2020 MARVEL project framework design activities, certain similarities and overlaps were identified between the functionalities of StreamHandler and those of DFB and DatAna with regards to big data management. However, following an in-depth analysis of the H2020 MARVEL project framework requirements that the DMP should satisfy, a gap was identified that could not be covered by the DFB and DatAna solutions. This gap was related to the management of audio-visual data. More specifically, the following requirements were established:  
* Receive and efficiently archive live streams of audiovisual binary data from all the relevant H2020 MARVEL project sensors, devices, and components during system operation. * The persistent storage of archived AV data should comply with high data security standards and data privacy requirements. * Provide access to archived audiovisual binary data to the H2020 MARVEL project UI (SmartViz) by streaming requested archived audiovisual data upon demand in order to present them to the end-user and in association with relevant inference results produced by H2020 MARVEL project AI components. * Support the expansion of the data set of the Data Corpus by relaying selected archived audiovisual data to it. StreamHandler was found to be in a position to be able to satisfy these requirements and fill the gap by extending its supported data source types, its connectors, and data storage capabilities. This course of action was aligned with INTRA's strategic plan to expand the StreamHandler platform in the direction of audiovisual data management for increased interoperability in order to address additional business cases and reinforce its position in big data management and smart city domains. The H2020 MARVEL project Data Corpus-as-a-Service has been one of the major outcomes of the H2020 MARVEL project trying to address the lack of extremely large public sets of annotated audio-visual recordings, which has been an obstacle for the scientific and industrial community to enhance audio-visual analytics. The H2020 MARVEL project Data Corpus is an extreme-scale public corpus of processed multimodal audio-visual open data, obtained free of charge and released as a service under the aegis of a specific service level agreement (SLA). It addresses all aspects that are related to the efficient sharing of heterogeneous data pools such as accessibility, operability, managing streaming and network, legal considerations, security, privacy, and technical concerns. The H2020 MARVEL project Data Corpus-as-a-Service has been the user's endpoint for accessing, through respective queries, the vast processed audio-visual data that is being stored in the H2020 MARVEL project infrastructure. Enriching and sharing the Data Corpus drives research and innovation in multimodal processing and analytics, which contribute to the process of benchmarking edge, fog, cloud, and AI technologies. Furthermore, besides its contribution to the areas of open science and open data, the H2020 MARVEL project Data Corpus-as-a-Service empowers smart city authorities to better support their societies, deriving new and advancing the existing knowledge; it enables them to build and deploy innovative applications that are based on multimodal perception and intelligence. Furthermore, the H2020 MARVEL project Data Corpus-as-a-Service serves as a key source of interaction with developers and researchers. It serves as a pool of data available to the public, enabling the scientific and industrial community to enhance audio-visual analytics and enforcing the foundations for smart city services that improve the security and well-being of their citizens. In addition, the H2020 MARVEL project Data Corpus provides the possibility for SMEs and start-ups to build on top of these data assets and create new business by exploring extreme-scale multimodal analytics.