Reviewerly/exHarmony huggingface dataset:
Deep Learning Approaches on Multimodal Sentiment Analysis

Sentiment analysis aims to uncover people&#x0027;s sentiment based on some information about them, often using machine learning or deep learning algorithm to determine. The importance of such a technique heavily grows because it can help companies better understand users&#x0027; attitudes toward things and decide future plans. Considering the maturity of sentiment analysis on a single modal such as text and image and the advancement of social media, which allow users to post with text and image simultaneously, novel methods are designed for multimodal sentiment analysis. Because of the infancy of this field, new methods are sometimes inefficient, unstable and lack robustness and still need further study. One big challenge on the multimodal sentiment analysis is to find a proper way that can overcome heterogeneous gap between different modalities, which implies that researchers should develop a method of combining various modalities properly to get higher efficiency and accuracy. Although much research has been performed and many new models are implemented, related summary is still lacking. This paper mainly introduces four interesting and relatively novel methods of solving the links among diverse modalities. First it briefly introduces some basic definitions and terminologies in sentiment analysis, followed by several popular and common datasets on multimodal sentiment analysis. After that it gives some evaluation index to assess one model&#x0027;s competence. Then it shows four models and compares pros and cons among them, showing the potential trend of future study in multimodal sentiment analysis. Finally, it proposes future challenges and opportunities.
ECM-ILTP: An Efficient Classification model for categorization of Interstitial Lung Tissue Patterns

Interstitial lung diseases (ILD) comprise various anomalous patterns examined in high-resolution computed tomography (HRCT) images. The precise interpretation of lung tissue patterns plays a very important role in clinical diagnosis. Therefore, it becomes a necessity to build accurate computer-aided detection (CAD) systems. This paper presents a methodology to categorize commonly found lung tissue patterns of ILD in HRCT (viz. fibrosis, ground glass, emphysema, healthy, and micronodules). The proposed methodology uses intensity-based, texture-based, and morphology-based shape features. The inclusion of shape-based features helps to conquer the inter-class and intra-class variation amongst the lung tissue patterns. Later, a Random Forest algorithm is used to classify lung tissue patterns. A publicly available benchmarked TALISMAN dataset of the ILD database of HUG is used to evaluate the proposed methodology. The performance of the proposed methodology is analyzed by using accuracy, precision, recall, f-measure, and normalized confusion matrix. The obtained performance measures of the proposed methodology, i.e., accuracy, f1-score, precision, and recall, are 0.858, 0.822, 0.835, and 0.817, respectively, which confirms its superiority and efficiency in categorizing lung tissue patterns over state-of-the-art methods.
Design of a VR-Based Campus Tour Platform with a User-Friendly Scene Asset Management System

Virtual reality tours have become a desire for many educational institutions due to the potential difficulties for students to attend in person, especially during the COVID-19 pandemic. Having the ability for a student to explore the buildings and locations on a university campus is a crucial part of convincing them to enroll in classes. Many institutions have already installed tour applications that they have either designed themselves or contracted to a third party. However, they lack a convenient way for non-maintainers, such as faculty, to manage and personalize their classrooms and offices in a simple way. In this paper, we propose a platform to not only provide a full virtual experience of the campus but also feature a user-friendly content management system designed for staff and faculty to customize their assigned scenes. The tour uses the Unity3D engine, which communicates to a university server hosting a custom .NET API and SQL database to obtain information about the virtual rooms through a role-based access system to the faculty and staff. We believe this system for managing tour scenes will solve both time and expense for the tour development team and allow them to focus on implementing other features, rather than having to fulfill requests for editing locations in the tour. We expect this framework to function as a tour platform for other universities, as well as small businesses and communities. We seek to demonstrate the feasibility of this platform through our developed prototype application. Based on small sample testing, we have received overall positive responses and constructive critique that has played a role in improving the application moving forward.
Comparing Decentralized Algorithms for Dynamic Task Sharing among Agents with Limited Resources

Dynamic task sharing among organizations is attracting attention in research as the COVID-19 pandemic continues and many hospitals are filled with patients. Here, the task is, for example, the treatment of patients requiring resources such as beds and medical staffs for certain periods of time. This task must be started before its completion becomes overdue. However, because these tasks are created dynamically, the task-handling cannot be scheduled in advance; in addition, these tasks are handled by multiple independent organizations. Because the distribution of tasks are not uniform across the organizations, some tasks should be transferred to unoccupied organizations. In this study, we present and compare some decentralized algorithms for dynamic task sharing among organizations with limited resources. We aim to minimize the number of unstarted tasks within the time limit.
Detection of Maliciously Blurred Image Portions Using Dyadic Wavelet Transform and Jensen-Shannon Divergence

The aim of this study is to detect malicious blurred areas in an image using dyadic wavelet transform, wavelet transform mod-ulus maxima, and Jensen-Shannon divergence. Nowadays, with the popularity of electronic devices, anyone can easily tamper images. The proposed method detects malicious blurring in areas of an image by blurring the edges of the tampered areas to copy-move and splicing. The dyadic wavelet transform is applied to the image, and the Jensen-Shannon divergence is computed for low-frequency components and luminance distribution of the original image to detect the suspected blurred regions. The experimental results show that the proposed method can detect suspected ma-licious blurs with an accuracy of 86.2 % are few wrong classifications.
Sentiment Guided Aspect Conditioned Dialogue Generation in a Multimodal System

Multimodality in conversations has become critical for gaining a complete grasp of the user's intention and providing better replies to satisfy the requirements of the customers. Existing multimodal conversational systems suffer from contradictions and generic responses. User sentiments upon the different aspects of a product/service are essential to comprehend the needs of the user and respond in an informative and interactive manner. In this regard, we propose the novel task of sentiment-guided aspect controlled response generation. This task is introduced to ensure consistency and coherence with the sentiments of the users for the aspects mentioned in the ongoing dialogue for generating better responses. In our work, we design a generative framework that utilizes the sentiment information of the previous utterances in a reinforced hierarchical transformer-based network. The decoder is provided the aspect knowledge explicitly for generation. We devise task-specific rewards that guide the generation process in an end-to-end manner. The multi-domain multi-modal conversation (MDMMD) dataset, which includes both text and images, is used to validate our proposed architecture. Quantitative and qualitative analyses show that the proposed network generates consistent and diverse responses, and performs superior to the existing frameworks.
An Efficient User’s Attribute Revocation Scheme Suitable for Data Outsourcing in Cloud Storage

With the advances of cloud data centers and cloud service, many application scenarios are developed such as enterprise business, the Vehicular Ad Hoc Networks (VANET), Sensor network. Confidentiality and efficiency are two key problems. CP-ABE is one of fine-grained access control cryptographic technologies and it is widely applied in outsourced data in cloud storage to protect the user’s privacy. In addition, besides confidentiality, computational cost is an important factor which makes the application of CP-ABE possible in the situations. In this paper, we propose a novel and fast scheme based on CP-ABE algorithm from the respect of the user’s attribute revocation to make it faster. In this scheme, we construct an identical tree T2 that has the same structure as the actual access control tree T1. When the user issues to request the encrypted text preserved on CSP, CSP first tries to decrypt CT2 which is encrypted according to the access policy based on the identical tree T2 and also preserved on it, with the subset of the secret keys corresponding to the indexes of the attributes provided by the user. If CSP can successfully decrypt it, it proves that he/she has the authorization to access the cryptographic text CT1 and CSP will send CT1 to the user. Otherwise, CSP recognizes that the user has no access to CT1 and rejects to send CT1 to the user. Namely, we can decide if we authorize the right to access CT1 by judging if the secret key provided by the user can decrypt CT2. Because CT1 and CT2 have the same access control structure, they have the same secret key for decryption; that is, when some attributes are revoked, if the secret key cannot decrypt CT2, it cannot decrypt CT1. While CSP can directly decrypt CT2, but not CT1, CSP can judge if the user has access to CT1 after the attribute revocation by decryption of CT2. Moreover, we propose the construction method of CT1 and CT2. Finally, we prove that the scheme is secure and viable.
FMFCC-A: A Challenging Mandarin Dataset for Synthetic Speech Detection

As increasing development of text-to-speech (TTS) and voice conversion (VC) technologies, the detection of synthetic speech has been suffered dramatically. In order to promote the development of synthetic speech detection model against Mandarin TTS and VC technologies, we have constructed a challenging Mandarin dataset and organized the accompanying audio track of the first fake media forensic challenge of China Society of Image and Graphics (FMFCC-A). The FMFCC-A dataset is by far the largest publicly-available Mandarin dataset for synthetic speech detection, which contains 40,000 synthesized Mandarin utterances that generated by 11 Mandarin TTS systems and two Mandarin VC systems, and 10,000 genuine Mandarin utterances collected from 58 speakers. The FMFCC-A dataset is divided into the training, development and evaluation sets, which are used for the research of detection of synthesized Mandarin speech under various previously unknown speech synthesis systems or audio post-processing operations. In addition to describing the construction of the FMFCC-A dataset, we provide a detailed analysis of two baseline methods and the top-performing submissions from the FMFCC-A, which illustrates the usefulness and challenge of FMFCC-A dataset. We hope that the FMFCC-A dataset can fill the gap of lack of Mandarin datasets for synthetic speech detection.
Ultrasonic Thyroid Automatic Nodule Segmentation Method Based on Sevnet Network

Background: Thyroid nodules are one of the most common cancers in the world. Ultrasound thyroid nodule imaged segmentation are of great significance for the clinical diagnosis and evaluation . However, the current standard of cancerous nodules in ultrasound imaging is not perfect, and the changes in the echo pattern of thyroid nodules will greatly limit the diagnostic ability of doctors, making nodule segmentation difficult. In this study, we employed the SEVnet network for automatic segmentation of thyroid nodule ultrasound images.Methods: In this paper, we combine the base Vnet network and the Squeeze-and-Excitation block, from which we propose an improved Vnet called SEVnet. Using SE block, the network can explicitly model the interdependence between feature channels, analyze the importance of each feature channel and assign weights by itself so that the network can recover the details well for segmentation. This is meaningful for accurate segmentation of thyroid nodules.Results: The proposed SEVnet model is implemented on a combined dataset of thyroid ultrasound images from the DDTI public database and from Hunan Provincial People's Hospital. The SEVnet network achieves an average accuracy of 91.783% and an average dice loss of 95.695 on this dataset. We compare the various indicators of the proposed methods and other advanced methods and find our proposed method has performed better in the indicators we use. In addition, we also directly compare the segmentation results of the proposed method and the above models, and the results show that our proposed network has some unique advantages.
A Secure Data Storage in Multi-cloud Architecture Using Blowfish Encryption Algorithm

In recent years, cloud computing has played an important role in accessing services around the world. Thanks to its rapid development in the IT industry it has gained huge appeal. It offers users the ability to store, retrieve and share data and resources with other users however and whenever they wish. Availability, confidentiality and integrity are key issues in cloud computing. The open nature of the technology and public access make data and applications in a single cloud vulnerable to multiple security breaches such as single point of failure and malicious insiders. To overcome such issues, the data is stored in a multi-cloud environment. In this paper, we integrate different security approaches in order to provide a new lightweight security framework for authentication and data storage in multi-cloud environment using three main algorithms: certificate authentication for user identification, Blowfish for data encryption, and RSA for the transfer of Blowfish's secret key. The proposed solution is evaluated based on download time, upload time and access time. Analysis of the experimental results shows that the proposed architecture guarantees secure, fast and lightweight access to data while preserving data confidentiality against unauthorized third parties.
Making Toll Charges Collection Efficient and Trustless: A Blockchain-Based Approach

The paper attempted to combine the road toll charges collection systems and blockchain technology to examine how well these two concepts could fit together in terms of better functioning and security. Toll plazas in India have a colossal rush and are always questionable for their service and the actual amount of payment collection. To avoid the hustle and cash payment in the current situation, go for the e-payment system. This will really help to avoid carrying cash amount as well change of money. But the constraints are required internet for the execution of e payment system. In this system, we need trend people, standard procedures for the payment system, and enhanced technologies. An entire financial exchange is based on a bank and its network link. But the bank has a centralized system, and sometimes it is not secure. This research work proposed the toll payment system based on blockchain to provide a safer and manageable system. As the blockchain feature, this proposed model is secure and least chance to temple the data. In the working process of this system, we use mobile to read QR codes at the toll plaza for the payment. The system uses a smart contract to pay as per the traveling distance covered by the vehicle, and every toll system has a record of these transactions.
Variational Monocular Depth Estimation for Reliability Prediction

Self-supervised learning for monocular depth estimation has been widely investigated as an alternative to the supervised learning approach. Uncertainty estimation in depth estimation is a crucial problem for applications, such as autonomous driving, in detecting unreliable depth. In this study, we propose a variational model to estimate depth uncertainty in self-supervised learning. Our approach leverages time-series images to handle the depth distribution from appearance variations in training. We introduce the Mahalanobis-Wasserstein distance between two consecutive frames to learn the uncertainty. In inference, our method estimates the uncertainty map and the depth image at each pixel from a single image. In experiments on KITTI, Make3D, and DIODE datasets, we show that our model achieves better uncertainty estimation than previous approaches as well as high accuracy of depth estimation
Graph Convolutional Neural Network with Inter-layer Cascade Based on Attention Mechanism

In recent years, graph data in the non-Euclidean space has been widely used, and the methods and techniques for learning graph data in many deep learning fields have been continuously developed, such as the Graph neural network (GNN). The structural characteristics of graph data, the aggregation mode and representation mode of node information, and the node neighbor information are the core issues of GNN. However, most of the existing graph convolutional neural networks have an excessive smoothing problem, which limits the learning ability of the model. Aiming at the over-smoothing problem of the current algorithm, this paper enhances the learning of graph data by improving the expression ability of local information and global features. This paper constructs a cascade structure between graph convolutional layers. This kind of network structure realizes the dense connection of convolutional layers, makes the local feature information is effectively used, and further enhances the graph representation ability. Introduce self-attention and TopK into the Readout module, selectively aggregate and express feature information, and use graph-level information more efficiently. Graph classification is a downstream task to verify the performance of the proposed model. Experimental results prove that this densely structured graph convolutional network can effectively aggregate local node information and global graph-level information.
Design of Sponsored Search Auction Mechanism for Federated Learning Advertising Platform

Federated learning has demonstrated strong capabilities in terms of addressing concerns related to data islands and privacy protection. However, in real application scenarios, participants in federated learning have difficulty matching. For example, two companies distributed in different regions do not know that the other party also needs federated learning in the case of information asymmetry. Therefore, it is difficult to build alliances. To enable suppliers and consumers to find one or more federated learning objects that are relatively satisfactory in a short time, this paper considers the idea of establishing a federated learning advertising platform, where data transactions need to consider privacy protection. A sponsored search auction mechanism design method is introduced to solve the problem of ranking the presentation order of participant advertisements. Due to the potential malicious bidding problem, which occurs when using the classic sponsored search auction mechanism under the federated learning scenario, this paper proposes a novel federated sponsored search auction mechanism based on the Myerson theorem, improving upon the ranking index used in the classic sponsored search auction mechanism. A large number of experimental results on a simulation data set show that our proposed method can fairly select and rank the data providers participating in the bidding. Compared with other benchmark mechanisms, the malicious bidding rate is significantly decreased. In the long run, the proposed mechanism can encourage more data providers to participate in the federated learning platform, thus continuously promoting the establishment of a federated learning ecosystem.
Self-Sovereign Identity and User Control for Privacy-Preserving Contact Tracing

Contact tracing apps use mobile devices to keep track of and promptly identify those who come in contact with an individual who tests positive for COVID-19. However, privacy is a major obstacle to the wide-spread use of such apps since users are concerned about sharing their contact and diagnosis data. This research overcomes multiple challenges facing contact tracing apps: (1) As researchers have pointed out, there is a need to balance contact tracing effectiveness with the amount of user identity and diagnosis information shared. (2) No matter what information the user chooses to share, the app should safeguard the privacy of user information. (3) On the other hand, some essential test result information must be shared for the contact tracing app to work. While contact tracing apps have done a good job maintaining contact information on the user’s device, most such apps publish positive COVID-19 test results to a central server which have some risks for compromise. We address these challenges by (1) giving the user the right to choose how much information to share about their diagnosis and their identity, (2) building our novel contact tracing app on top of Self-Sovereign Identity (SSI) to assure privacy preserving user authentication with verifiable credentials, and (3) decentralizing the storage of COVID-19 test results. We, in collaboration with Verizon, have implemented our Privacy-preserving Contact Tracing (PpCT) app, leveraging SSI advances based on the blockchain for their 5G network.
Identification of Vehicle Trajectory Anomalies on Streaming Video

We suggested the approach for identifying a vehicle’s trajectory anomalies on streaming video. Video surveillance cameras installed on street intersections in Kazan city were used as the source data. We applied YOLO v3 artificial neural network and KCF object tracking algorithm to get an initial vehicle’s trajectories from raw video streams. Then we used polynomial approximation, thinning, and clustering for training our model and classifying all trajectories into normal or anomaly. The Ramer-Douglas-Peucker algorithm was used for trajectory’s thinning. We suggested the modified Longest Common SubSequence (LCSS) metric for the comparison of trajectories under spatial perspective. In the training stage, we used the agglomerative hierarchical clustering for the creation of two types of clusters – with normal and anomaly trajectories. In the classification stage, we used cluster models and the modified LCSS metric to classify certain vehicle trajectories as normal or abnormal. The framework has been developed for the identification of vehicle trajectory anomalies on video streams. This framework could be used as the key element of intelligent transport systems.