OpenRaiser/Intern-Atlas huggingface dataset:
Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains
Venue: arXiv cs.CL 2022
Year: 2022

Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains. However, low-resource settings such as new and emerging domains would especially benefit from reliable question answering systems. Furthermore, multilingual and cross-lingual resources in emergent domains are scarce, leading to few or no such systems. In this paper, we demonstrate a cross-lingual open-retrieval question answering system for the emergent domain of COVID-19. Our system adopts a corpus of scientific articles to ensure that retrieved documents are reliable. To address the scarcity of cross-lingual training data in emergent domains, we present a method utilizing automatic translation, alignment, and filtering to produce English-to-all datasets. We show that a deep semantic retriever greatly benefits from training on our English-to-all data and significantly outperforms a BM25 baseline in the cross-lingual setting. We illustrate the capabilities of our system with examples and release all code necessary to train and deploy such a system.
Explainable Patterns for Distinction and Prediction of Moral Judgement on Reddit
Venue: arXiv cs.CL 2022
Year: 2022

The forum r/AmITheAsshole in Reddit hosts discussion on moral issues based on concrete narratives presented by users. Existing analysis of the forum focuses on its comments, and does not make the underlying data publicly available. In this paper we build a new dataset of comments and also investigate the classification of the posts in the forum. Further, we identify textual patterns associated with the provocation of moral judgement by posts, with the expression of moral stance in comments, and with the decisions of trained classifiers of posts and comments.
DiGamma: Domain-aware Genetic Algorithm for HW-Mapping Co-optimization for DNN Accelerators
Venue: arXiv cs.NE 2022
Year: 2022

The design of DNN accelerators includes two key parts: HW resource configuration and mapping strategy. Intensive research has been conducted to optimize each of them independently. Unfortunately, optimizing for both together is extremely challenging due to the extremely large cross-coupled search space. To address this, in this paper, we propose a HW-Mapping co-optimization framework, an efficient encoding of the immense design space constructed by HW and Mapping, and a domain-aware genetic algorithm, named DiGamma, with specialized operators for improving search efficiency. We evaluate DiGamma with seven popular DNNs models with different properties. Our evaluations show DiGamma can achieve (geomean) 3.0x and 10.0x speedup, comparing to the best-performing baseline optimization algorithms, in edge and cloud settings.
An Exploration of Captioning Practices and Challenges of Individual Content Creators on YouTube for People with Hearing Impairments
Venue: arXiv cs.HC 2022
Year: 2022

Deaf and Hard-of-Hearing (DHH) audiences have long complained about caption qualities for many online videos created by individual content creators on video-sharing platforms (e.g., YouTube). However, there lack explorations of practices, challenges, and perceptions of online video captions from the perspectives of both individual content creators and DHH audiences. In this work, we first explore DHH audiences' feedback on and reactions to YouTube video captions through interviews with 13 DHH individuals, and uncover DHH audiences' experiences, challenges, and perceptions on watching videos created by individual content creators (e.g., manually added caption tags could create additional confidence and trust in caption qualities for DHH audiences). We then discover individual content creators' practices, challenges, and perceptions on captioning their videos (e.g., back-captioning problems) by conducting a YouTube video analysis with 189 captioning-related YouTube videos, followed by a survey with 62 individual content creators. Overall, our findings provide an in-depth understanding of captions generated by individual content creators and bridge the knowledge gap mutually between content creators and DHH audiences on captions.
Jointly Learning Knowledge Embedding and Neighborhood Consensus with Relational Knowledge Distillation for Entity Alignment
Venue: arXiv cs.LG 2022
Year: 2022

Entity alignment aims at integrating heterogeneous knowledge from different knowledge graphs. Recent studies employ embedding-based methods by first learning the representation of Knowledge Graphs and then performing entity alignment via measuring the similarity between entity embeddings. However, they failed to make good use of the relation semantic information due to the trade-off problem caused by the different objectives of learning knowledge embedding and neighborhood consensus. To address this problem, we propose Relational Knowledge Distillation for Entity Alignment (RKDEA), a Graph Convolutional Network (GCN) based model equipped with knowledge distillation for entity alignment. We adopt GCN-based models to learn the representation of entities by considering the graph structure and incorporating the relation semantic information into GCN via knowledge distillation. Then, we introduce a novel adaptive mechanism to transfer relational knowledge so as to jointly learn entity embedding and neighborhood consensus. Experimental results on several benchmarking datasets demonstrate the effectiveness of our proposed model.
Wireless IoT Energy Sharing Platform
Venue: arXiv cs.DC 2022
Year: 2022

Wireless energy sharing is a novel convenient alternative to charge IoT devices. In this demo paper, we present a peer-to-peer wireless energy sharing platform. The platform enables users to exchange energy wirelessly with nearby IoT devices. The energy sharing platform allows IoT users to send and receive energy wirelessly. The platform consists of (i) a mobile application that monitors and synchronizes the energy transfer among two IoT devices and (ii) and a backend to register energy providers and consumers and store their energy transfer transactions. The eveloped framework allows the collection of a real wireless energy sharing dataset. A set of preliminary experiments has been conducted on the collected dataset to analyze and demonstrate the behavior of the current wireless energy sharing technology.
Hierarchical Visual Interface for Lecture Video Retrieval and Summarization
Venue: arXiv cs.HC 2022
Year: 2022

With the emergence of large-scale open online courses and online academic conferences, it has become increasingly feasible and convenient to access online educational resources. However, it is time consuming and challenging to effectively retrieve and present numerous lecture videos for common users. In this work, we propose a hierarchical visual interface for retrieving and summarizing lecture videos. Users can utilize the proposed interface to effectively explore the required video information through the results of the video summary generation in different layers. We retrieve the input keywords with the corresponding video layer with timestamps, a frame layer with slides, and the poster layer with summarization of the lecture videos. We verified the proposed interface with our user study by comparing it with other conventional interfaces. The results from our user study confirmed that the proposed interface can achieve high retrieval accuracy and good user experience.see video here https://www.youtube.com/watch?v=zrnejwsOVpc .
Highly Generalizable Models for Multilingual Hate Speech Detection
Venue: arXiv cs.CL 2022
Year: 2022

Hate speech detection has become an important research topic within the past decade. More private corporations are needing to regulate user generated content on different platforms across the globe. In this paper, we introduce a study of multilingual hate speech classification. We compile a dataset of 11 languages and resolve different taxonomies by analyzing the combined data with binary labels: hate speech or not hate speech. Defining hate speech in a single way across different languages and datasets may erase cultural nuances to the definition, therefore, we utilize language agnostic embeddings provided by LASER and MUSE in order to develop models that can use a generalized definition of hate speech across datasets. Furthermore, we evaluate prior state of the art methodologies for hate speech detection under our expanded dataset. We conduct three types of experiments for a binary hate speech classification task: Multilingual-Train Monolingual-Test, MonolingualTrain Monolingual-Test and Language-Family-Train Monolingual Test scenarios to see if performance increases for each language due to learning more from other language data.
Network Slicing with MEC and Deep Reinforcement Learning for the Internet of Vehicles
Venue: arXiv cs.NI 2022
Year: 2022

The interconnection of vehicles in the future fifth generation (5G) wireless ecosystem forms the so-called Internet of vehicles (IoV). IoV offers new kinds of applications requiring delay-sensitive, compute-intensive and bandwidth-hungry services. Mobile edge computing (MEC) and network slicing (NS) are two of the key enabler technologies in 5G networks that can be used to optimize the allocation of the network resources and guarantee the diverse requirements of IoV applications. As traditional model-based optimization techniques generally end up with NP-hard and strongly non-convex and non-linear mathematical programming formulations, in this paper, we introduce a model-free approach based on deep reinforcement learning (DRL) to solve the resource allocation problem in MEC-enabled IoV network based on network slicing. Furthermore, the solution uses non-orthogonal multiple access (NOMA) to enable a better exploitation of the scarce channel resources. The considered problem addresses jointly the channel and power allocation, the slice selection and the vehicles selection (vehicles grouping). We model the problem as a single-agent Markov decision process. Then, we solve it using DRL using the well-known DQL algorithm. We show that our approach is robust and effective under different network conditions compared to benchmark solutions.
Towards a Scalable and Trustworthy Blockchain: IoT Use Case
Venue: arXiv cs.CR 2022
Year: 2022

Recently, blockchain has gained momentum as a novel technology that gives rise to a plethora of new decentralized applications (e.g., Internet of Things (IoT)). However, its integration with the IoT is still facing several problems (e.g., scalability, flexibility). Provisioning resources to enable a large number of connected IoT devices implies having a scalable and flexible blockchain. To address these issues, we propose a scalable and trustworthy blockchain (STB) architecture that is suitable for the IoT; which uses blockchain sharding and oracles to establish trust among unreliable IoT devices in a fully distributed and trustworthy manner. In particular, we design a Peer-To-Peer oracle network that ensures data reliability, scalability, flexibility, and trustworthiness. Furthermore, we introduce a new lightweight consensus algorithm that scales the blockchain dramatically while ensuring the interoperability among participants of the blockchain. The results show that our proposed STB architecture achieves flexibility, efficiency, and scalability making it a promising solution that is suitable for the IoT context.
Towards Agnostic Feature-based Dynamic Pricing: Linear Policies vs Linear Valuation with Unknown Noise
Venue: arXiv cs.LG 2022
Year: 2022

In feature-based dynamic pricing, a seller sets appropriate prices for a sequence of products (described by feature vectors) on the fly by learning from the binary outcomes of previous sales sessions ("Sold" if valuation $\geq$ price, and "Not Sold" otherwise). Existing works either assume noiseless linear valuation or precisely-known noise distribution, which limits the applicability of those algorithms in practice when these assumptions are hard to verify. In this work, we study two more agnostic models: (a) a "linear policy" problem where we aim at competing with the best linear pricing policy while making no assumptions on the data, and (b) a "linear noisy valuation" problem where the random valuation is linear plus an unknown and assumption-free noise. For the former model, we show a $\tildeΘ(d^{\frac13}T^{\frac23})$ minimax regret up to logarithmic factors. For the latter model, we present an algorithm that achieves an $\tilde{O}(T^{\frac34})$ regret, and improve the best-known lower bound from $Ω(T^{\frac35})$ to $\tildeΩ(T^{\frac23})$. These results demonstrate that no-regret learning is possible for feature-based dynamic pricing under weak assumptions, but also reveal a disappointing fact that the seemingly richer pricing feedback is not significantly more useful than the bandit-feedback in regret reduction.
Prediction and Detection of FDIA and DDoS Attacks in 5G Enabled IoT
Venue: arXiv cs.CR 2022
Year: 2022

Security in the fifth generation (5G) networks has become one of the prime concerns in the telecommunication industry. 5G security challenges come from the fact that 5G networks involve different stakeholders using different security requirements and measures. Deficiencies in security management between these stakeholders can lead to security attacks. Therefore, security solutions should be conceived for the safe deployment of different 5G verticals (e.g., industry 4.0, Internet of Things (IoT), etc.). The interdependencies among 5G and fully connected systems, such as IoT, entail some standard security requirements, namely integrity, availability, and confidentiality. In this article, we propose a hierarchical architecture for securing 5G enabled IoT networks, and a security model for the prediction and detection of False Data Injection Attacks (FDIA) and Distributed Denial of Service attacks (DDoS). The proposed security model is based on a Markov stochastic process, which is used to observe the behavior of each network device, and employ a range-based behavior sifting policy. Simulation results demonstrate the effectiveness of the proposed architecture and model in detecting and predicting FDIA and DDoS attacks in the context of 5G enabled IoT.
An IoT Blockchain Architecture Using Oracles and Smart Contracts: the Use-Case of a Food Supply Chain
Venue: arXiv cs.NI 2022
Year: 2022

The blockchain is a distributed technology which allows establishing trust among unreliable users who interact and perform transactions with each other. While blockchain technology has been mainly used for crypto-currency, it has emerged as an enabling technology for establishing trust in the realm of the Internet of Things (IoT). Nevertheless, a naive usage of the blockchain for IoT leads to high delays and extensive computational power. In this paper, we propose a blockchain architecture dedicated to being used in a supply chain which comprises different distributed IoT entities. We propose a lightweight consensus for this architecture, called LC4IoT. The consensus is evaluated through extensive simulations. The results show that the proposed consensus uses low computational power, storage capability and latency.
CacheFX: A Framework for Evaluating Cache Security
Venue: arXiv cs.CR 2022
Year: 2022

Over the last two decades, the danger of sharing resources between programs has been repeatedly highlighted. Multiple side-channel attacks, which seek to exploit shared components for leaking information, have been devised, mostly targeting shared caching components. In response, the research community has proposed multiple cache designs that aim at curbing the source of side channels. With multiple competing designs, there is a need for assessing the level of security against side-channel attacks that each design offers. In this work we propose CacheFX, a flexible framework for assessing and evaluating the resilience of cache designs to side-channel attacks. CacheFX allows the evaluator to implement various cache designs, victims, and attackers, as well as to exercise them for assessing the leakage of information via the cache. To demonstrate the power of CacheFX, we implement multiple cache designs and replacement algorithms, and devise three evaluation metrics that measure different aspects of the caches:(1) the entropy induced by a memory access; (2) the complexity of building an eviction set; and (3) protection against cryptographic attacks. Our experiments highlight that different security metrics give different insights to designs, making a comprehensive analysis mandatory. For instance, while eviction-set building was fastest for randomized skewed caches, these caches featured lower eviction entropy and higher practical attack complexity. Our experiments show that all non-partitioned designs allow for effective cryptographic attacks. However, in state-of-the-art secure caches, eviction-based attacks are more difficult to mount than occupancy-based attacks, highlighting the need to consider the latter in cache design.
Achieving Personalized Federated Learning with Sparse Local Models
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning (FL) is vulnerable to heterogeneously distributed data, since a common global model in FL may not adapt to the heterogeneous data distribution of each user. To counter this issue, personalized FL (PFL) was proposed to produce dedicated local models for each individual user. However, PFL is far from its maturity, because existing PFL solutions either demonstrate unsatisfactory generalization towards different model architectures or cost enormous extra computation and memory. In this work, we propose federated learning with personalized sparse mask (FedSpa), a novel PFL scheme that employs personalized sparse masks to customize sparse local models on the edge. Instead of training an intact (or dense) PFL model, FedSpa only maintains a fixed number of active parameters throughout training (aka sparse-to-sparse training), which enables users' models to achieve personalization with cheap communication, computation, and memory cost. We theoretically show that the iterates obtained by FedSpa converge to the local minimizer of the formulated SPFL problem at rate of $\mathcal{O}(\frac{1}{\sqrt{T}})$. Comprehensive experiments demonstrate that FedSpa significantly saves communication and computation costs, while simultaneously achieves higher model accuracy and faster convergence speed against several state-of-the-art PFL methods.
The MSXF TTS System for ICASSP 2022 ADD Challenge
Venue: arXiv cs.SD 2022
Year: 2022

This paper presents our MSXF TTS system for Task 3.1 of the Audio Deep Synthesis Detection (ADD) Challenge 2022. We use an end to end text to speech system, and add a constraint loss to the system when training stage. The end to end TTS system is VITS, and the pre-training self-supervised model is wav2vec 2.0. And we also explore the influence of the speech speed and volume in spoofing. The faster speech means the less the silence part in audio, the easier to fool the detector. We also find the smaller the volume, the better spoofing ability, though we normalize volume for submission. Our team is identified as C2, and we got the fourth place in the challenge.
Report: State of the Art Solutions for Privacy Preserving Machine Learning in the Medical Context
Venue: arXiv cs.CR 2022
Year: 2022

Machine Learning on Big Data gets more and more attention in various fields. Even so privacy-preserving techniques become more important, even necessary due to legal regulations such as the General Data Protection Regulation (GDPR). On the other hand data is often distributed among various parties. Especially in the medical context there are several data holders, e.g. hospitals and we need to deal with highly sensitive values. A real world scenario would be data that is held in an electronic patient record that is available in many countries by now. The medical data is encrypted. Users (e.g. physicians, hospitals) can only decrypt the data after patient authorization. One of the main questions concerning this scenario is whether it is possible to process the data for research purposes without violating the privacy of the data owner. We want to evaluate which cryptographic mechanism - homomorphic encryption, multiparty computation or trusted execution environements - can be used for this task.
Non-linear Motion Estimation for Video Frame Interpolation using Space-time Convolutions
Venue: arXiv cs.CV 2022
Year: 2022

Video frame interpolation aims to synthesize one or multiple frames between two consecutive frames in a video. It has a wide range of applications including slow-motion video generation, frame-rate up-scaling and developing video codecs. Some older works tackled this problem by assuming per-pixel linear motion between video frames. However, objects often follow a non-linear motion pattern in the real domain and some recent methods attempt to model per-pixel motion by non-linear models (e.g., quadratic). A quadratic model can also be inaccurate, especially in the case of motion discontinuities over time (i.e. sudden jerks) and occlusions, where some of the flow information may be invalid or inaccurate. In our paper, we propose to approximate the per-pixel motion using a space-time convolution network that is able to adaptively select the motion model to be used. Specifically, we are able to softly switch between a linear and a quadratic model. Towards this end, we use an end-to-end 3D CNN encoder-decoder architecture over bidirectional optical flows and occlusion maps to estimate the non-linear motion model of each pixel. Further, a motion refinement module is employed to refine the non-linear motion and the interpolated frames are estimated by a simple warping of the neighboring frames with the estimated per-pixel motion. Through a set of comprehensive experiments, we validate the effectiveness of our model and show that our method outperforms state-of-the-art algorithms on four datasets (Vimeo, DAVIS, HD and GoPro).
An Analysis on Ensemble Learning optimized Medical Image Classification with Deep Convolutional Neural Networks
Venue: arXiv cs.CV 2022
Year: 2022

Novel and high-performance medical image classification pipelines are heavily utilizing ensemble learning strategies. The idea of ensemble learning is to assemble diverse models or multiple predictions and, thus, boost prediction performance. However, it is still an open question to what extent as well as which ensemble learning strategies are beneficial in deep learning based medical image classification pipelines. In this work, we proposed a reproducible medical image classification pipeline for analyzing the performance impact of the following ensemble learning techniques: Augmenting, Stacking, and Bagging. The pipeline consists of state-of-the-art preprocessing and image augmentation methods as well as 9 deep convolution neural network architectures. It was applied on four popular medical imaging datasets with varying complexity. Furthermore, 12 pooling functions for combining multiple predictions were analyzed, ranging from simple statistical functions like unweighted averaging up to more complex learning-based functions like support vector machines. Our results revealed that Stacking achieved the largest performance gain of up to 13% F1-score increase. Augmenting showed consistent improvement capabilities by up to 4% and is also applicable to single model based pipelines. Cross-validation based Bagging demonstrated significant performance gain close to Stacking, which resulted in an F1-score increase up to +11%. Furthermore, we demonstrated that simple statistical pooling functions are equal or often even better than more complex pooling functions. We concluded that the integration of ensemble learning techniques is a powerful method for any medical image classification pipeline to improve robustness and boost performance.