Social-sensor cloud services for scene reconstruction
Creators: name: AAMIR, Tooba
Publisher: name: RMIT University
Year: 2024
Subjects: classificationCode: 460911
subject: Inter-organisational, extra-organisational and global information systems

description: The rich and explosive growth of social media data has resulted in the integration of social data into a range of data-centric applications. Recent communication devices such as smartphones (social-sensors) allow sensor data to be embedded directly into cloud-based social networks, i.e., social-sensor clouds. The monitoring of these social-sensors¿ activities offers multiple benefits in various domains. For example, urban management requires the reconstruction and analysis of a scene in a particular area. For example, if the surveillance of a road segment by  traditional sensors is limited in terms of coverage, social-sensors can help to fill in any gaps in information pertaining to an event or occurrence.

Social-sensor data, such as social media image metadata and related posted data (e.g., location, description, and comments) are inherently multi-modal because of the different data formats and sources in social media platforms. The multifaceted data poses a significant challenge for the efficient and real-time delivery of social-sensor data to users. In this research, we propose social-sensor cloud services that offer an open, flexible, and reconfigurable platform for monitoring and controlling applications. We abstract social-sensor cloud data, such as image annotations (metadata and related information like description and comments) as a service, in other words a social-sensor cloud service, to fulfil the users¿ information requirements.

The aim of this research is to provide an efficient selection and composition of social-sensor cloud services comprising functional and non-functional features. We have investigated spatio-temporal, textual and other metadata features to select and compose social-sensor cloud services. Our aim is to design and develop a framework to compose social-sensor cloud services that can address user queries comprising multiple QoS criteria requirements. The novelty lies in utilising images and image meta-information to avoid using expensive traditional image processing for scene reconstruction. Metadata, such as the geolocation, time and perspective of an image are modelled as non-functional attributes of a social-sensor cloud service. In this research, we propose models for context-aware and context-free selection and composition.

In context-aware, social-sensor service composition, we propose a context and direction-aware spatio-temporal selection and composition approach. To reconstruct a scene, we use the spatio-temporal and textual features of social-sensor cloud services, i.e., GPS location, time, description, recognised tags and accompanying metadata. The novelty lies in proposing a context and direction-aware spatio-temporal clustering and recommendation approach for selecting a set of temporally and semantically similar services to compose the best available social-sensor cloud services.

In a context-free social-sensor service composition, we propose a heuristics and machine learning-based social-sensor cloud service selection and composition model to reconstruct mosaic scenes. The proposed approach automatically selects relevant and composable social- sensor cloud services based on the services¿ features and our designed heuristics. The proposed method is innovative as it applies a strategy to enable automatic social-sensor cloud service selection and composition for the analysis of scenes without using image processing and object-identification technologies. The proposed approach is appropriate for most social media platforms that have not fully realised image processing and object identification.

Moreover, we propose two novel trust models for establishing a trust rating in the social-sensor cloud service environment, taking into account its sparse and unpredictable nature. First, we propose a users¿ stance- and credibility-based social-sensor cloud service trust model. This model uses the textual features of the social-sensor cloud services, i.e., comments, and metadata such as spatio-temporal information, along with the users¿ credibility to determine the trustworthiness of the services. Second, we propose to utilise specific context-free and context-aware, non-functional attributes to assess the trustworthiness or authenticity of the social-sensor cloud services. Metadata, such as geolocation, time, camera-specific image properties, tags and description of a scene-related social media image are modelled as the non-functional attributes of a social-sensor cloud service.

All of the proposed approaches are innovative as they utilize  image meta-information to avoid using expensive, traditional image processing techniques to reconstruct scenes. To analyse and evaluate the performance of our proposed approaches, we use the scene reconstruction of an accident as an example. Analytical results based on real datasets are presented to demonstrate the performance of the proposed approaches.