OpenRaiser/Intern-Atlas huggingface dataset:
A Hinge-Loss based Codebook Transfer for Cross-Domain Recommendation with Nonoverlapping Data
Venue: arXiv cs.IR 2021
Year: 2021

Recommender systems(RS), especially collaborative filtering(CF) based RS, has been playing an important role in many e-commerce applications. As the information being searched over the internet is rapidly increasing, users often face the difficulty of finding items of his/her own interest and RS often provides help in such tasks. Recent studies show that, as the item space increases, and the number of items rated by the users become very less, issues like sparsity arise. To mitigate the sparsity problem, transfer learning techniques are being used wherein the data from dense domain(source) is considered in order to predict the missing entries in the sparse domain(target). In this paper, we propose a transfer learning approach for cross-domain recommendation when both domains have no overlap of users and items. In our approach the transferring of knowledge from source to target domain is done in a novel way. We make use of co-clustering technique to obtain the codebook (cluster-level rating pattern) of source domain. By making use of hinge loss function we transfer the learnt codebook of the source domain to target. The use of hinge loss as a loss function is novel and has not been tried before in transfer learning. We demonstrate that our technique improves the approximation of the target matrix on benchmark datasets.
Joint Active User Detection and Channel Estimation for Grant-Free NOMA-OTFS in LEO Constellation Internet-of-Things
Venue: arXiv cs.IT 2021
Year: 2021

The flourishing low-Earth orbit (LEO) constellation communication network provides a promising solution for seamless coverage services to Internet-of-Things (IoT) terminals. However, confronted with massive connectivity and rapid variation of terrestrial-satellite link (TSL), the traditional grant-free random-access schemes always fail to match this scenario. In this paper, a new non-orthogonal multiple-access (NOMA) transmission protocol that incorporates orthogonal time frequency space (OTFS) modulation is proposed to solve these problems. Furthermore, we propose a two-stages joint active user detection and channel estimation scheme based on the training sequences aided OTFS data frame structure. Specifically, in the first stage, with the aid of training sequences, we perform active user detection and coarse channel estimation by recovering the sparse sampled channel vectors. And then, we develop a parametric approach to facilitate more accurate result of channel estimation with the previously recovered sampled channel vectors according to the inherent characteristics of TSL channel. Simulation results demonstrate the superiority of the proposed method in this kind of high-mobility scenario in the end.
Domain Adaptor Networks for Hyperspectral Image Recognition
Venue: arXiv cs.CV 2021
Year: 2021

We consider the problem of adapting a network trained on three-channel color images to a hyperspectral domain with a large number of channels. To this end, we propose domain adaptor networks that map the input to be compatible with a network trained on large-scale color image datasets such as ImageNet. Adaptors enable learning on small hyperspectral datasets where training a network from scratch may not be effective. We investigate architectures and strategies for training adaptors and evaluate them on a benchmark consisting of multiple hyperspectral datasets. We find that simple schemes such as linear projection or subset selection are often the most effective, but can lead to a loss in performance in some cases. We also propose a novel multi-view adaptor where of the inputs are combined in an intermediate layer of the network in an order invariant manner that provides further improvements. We present extensive experiments by varying the number of training examples in the benchmark to characterize the accuracy and computational trade-offs offered by these adaptors.
Scheduling Aerial Vehicles in an Urban Air Mobility Scheme
Venue: arXiv cs.AI 2021
Year: 2021

Highly populated cities face several challenges, one of them being the intense traffic congestion. In recent years, the concept of Urban Air Mobility has been put forward by large companies and organizations as a way to address this problem, and this approach has been rapidly gaining ground. This disruptive technology involves aerial vehicles (AVs) for hire than can be utilized by customers to travel between locations within large cities. This concept has the potential to drastically decrease traffic congestion and reduce air pollution, since these vehicles typically use electric motors powered by batteries. This work studies the problem of scheduling the assignment of AVs to customers, having as a goal to maximize the serviced customers and minimize the energy consumption of the AVs by forcing them to fly at the lowest possible altitude. Initially, an Integer Linear Program (ILP) formulation is presented, that is solved offline and optimally, followed by a near-optimal algorithm, that solves the problem incrementally, one AV at a time, to address scalability issues, allowing scheduling in problems involving large numbers of locations, AVs, and customer requests.
Towards Substructural Property-Based Testing
Venue: arXiv cs.LO 2021
Year: 2021

We propose to extend property-based testing to substructural logics to overcome the current lack of reasoning tools in the field. We take the first step by implementing a property-based testing system for specifications written in the linear logic programming language Lolli. We employ the foundational proof certificates architecture to model various data generation strategies. We validate our approach by encoding a model of a simple imperative programming language and its compilation and by testing its meta-theory via mutation analysis.
An Interpretable Music Similarity Measure Based on Path Interestingness
Venue: arXiv cs.IR 2021
Year: 2021

We introduce a novel and interpretable path-based music similarity measure. Our similarity measure assumes that items, such as songs and artists, and information about those items are represented in a knowledge graph. We find paths in the graph between a seed and a target item; we score those paths based on their interestingness; and we aggregate those scores to determine the similarity between the seed and the target. A distinguishing feature of our similarity measure is its interpretability. In particular, we can translate the most interesting paths into natural language, so that the causes of the similarity judgements can be readily understood by humans. We compare the accuracy of our similarity measure with other competitive path-based similarity baselines in two experimental settings and with four datasets. The results highlight the validity of our approach to music similarity, and demonstrate that path interestingness scores can be the basis of an accurate and interpretable similarity measure.
Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translation
Venue: arXiv cs.CL 2021
Year: 2021

Multimodal target/aspect sentiment classification combines multimodal sentiment analysis and aspect/target sentiment classification. The goal of the task is to combine vision and language to understand the sentiment towards a target entity in a sentence. Twitter is an ideal setting for the task because it is inherently multimodal, highly emotional, and affects real world events. However, multimodal tweets are short and accompanied by complex, possibly irrelevant images. We introduce a two-stream model that translates images in input space using an object-aware transformer followed by a single-pass non-autoregressive text generation approach. We then leverage the translation to construct an auxiliary sentence that provides multimodal information to a language model. Our approach increases the amount of text available to the language model and distills the object-level information in complex images. We achieve state-of-the-art performance on two multimodal Twitter datasets without modifying the internals of the language model to accept multimodal data, demonstrating the effectiveness of our translation. In addition, we explain a failure mode of a popular approach for aspect sentiment analysis when applied to tweets. Our code is available at \textcolor{blue}{\url{https://github.com/codezakh/exploiting-BERT-thru-translation}}.
Improving Music Performance Assessment with Contrastive Learning
Venue: arXiv cs.SD 2021
Year: 2021

Several automatic approaches for objective music performance assessment (MPA) have been proposed in the past, however, existing systems are not yet capable of reliably predicting ratings with the same accuracy as professional judges. This study investigates contrastive learning as a potential method to improve existing MPA systems. Contrastive learning is a widely used technique in representation learning to learn a structured latent space capable of separately clustering multiple classes. It has been shown to produce state of the art results for image-based classification problems. We introduce a weighted contrastive loss suitable for regression tasks applied to a convolutional neural network and show that contrastive loss results in performance gains in regression tasks for MPA. Our results show that contrastive-based methods are able to match and exceed SoTA performance for MPA regression tasks by creating better class clusters within the latent space of the neural networks.
Improving Counterfactual Generation for Fair Hate Speech Detection
Venue: arXiv cs.CL 2021
Year: 2021

Bias mitigation approaches reduce models' dependence on sensitive features of data, such as social group tokens (SGTs), resulting in equal predictions across the sensitive features. In hate speech detection, however, equalizing model predictions may ignore important differences among targeted social groups, as hate speech can contain stereotypical language specific to each SGT. Here, to take the specific language about each SGT into account, we rely on counterfactual fairness and equalize predictions among counterfactuals, generated by changing the SGTs. Our method evaluates the similarity in sentence likelihoods (via pre-trained language models) among counterfactuals, to treat SGTs equally only within interchangeable contexts. By applying logit pairing to equalize outcomes on the restricted set of counterfactuals for each instance, we improve fairness metrics while preserving model performance on hate speech detection.
A Study on Herd Behavior Using Sentiment Analysis in Online Social Network
Venue: arXiv cs.SI 2021
Year: 2021

Social media platforms are thriving nowadays, so a huge volume of data is produced. As it includes brief and clear statements, millions of people post their thoughts on microblogging sites every day. This paper represents and analyze the capacity of diverse strategies to volumetric, delicate, and social networks to predict critical opinions from online social networking sites. In the exploration of certain searching for relevant, the thoughts of people play a crucial role. Social media becomes a good outlet since the last decades to share the opinions globally. Sentiment analysis as well as opinion mining is a tool that is used to extract the opinions or thoughts of the common public. An occurrence in one place, be it economic, political, or social, may trigger large-scale chain public reaction across many other sites in an increasingly interconnected world. This study demonstrates the evaluation of sentiment analysis techniques using social media contents and creating the association between subjectivity with herd behavior and clustering coefficient as well as tries to predict the election result (2021 election in West Bengal). This is an implementation of sentiment analysis targeted at estimating the results of an upcoming election by assessing the public's opinion across social media. This paper also has a short discussion section on the usefulness of the idea in other fields.
Scalable Community Detection via Parallel Correlation Clustering
Venue: arXiv cs.SI 2021
Year: 2021

Graph clustering and community detection are central problems in modern data mining. The increasing need for analyzing billion-scale data calls for faster and more scalable algorithms for these problems. There are certain trade-offs between the quality and speed of such clustering algorithms. In this paper, we design scalable algorithms that achieve high quality when evaluated based on ground truth. We develop a generalized sequential and shared-memory parallel framework based on the LambdaCC objective (introduced by Veldt et al.), which encompasses modularity and correlation clustering. Our framework consists of highly-optimized implementations that scale to large data sets of billions of edges and that obtain high-quality clusters compared to ground-truth data, on both unweighted and weighted graphs. Our empirical evaluation shows that this framework improves the state-of-the-art trade-offs between speed and quality of scalable community detection. For example, on a 30-core machine with two-way hyper-threading, our implementations achieve orders of magnitude speedups over other correlation clustering baselines, and up to 28.44x speedups over our own sequential baselines while maintaining or improving quality.
Deterministic Distributed Algorithms and Lower Bounds in the Hybrid Model
Venue: arXiv cs.DC 2021
Year: 2021

The $\hybrid$ model was recently introduced by Augustine et al. \cite{DBLP:conf/soda/AugustineHKSS20} in order to characterize from an algorithmic standpoint the capabilities of networks which combine multiple communication modes. Concretely, it is assumed that the standard $\local$ model of distributed computing is enhanced with the feature of all-to-all communication, but with very limited bandwidth, captured by the node-capacitated clique ($\ncc$). In this work we provide several new insights on the power of hybrid networks for fundamental problems in distributed algorithms. First, we present a deterministic algorithm which solves any problem on a sparse $n$-node graph in $\widetilde{\mathcal{O}}(\sqrt{n})$ rounds of $\hybrid$. We combine this primitive with several sparsification techniques to obtain efficient distributed algorithms for general graphs. Most notably, for the all-pairs shortest paths problem we give deterministic $(1 + ε)$- and $\log n/\log \log n$-approximate algorithms for unweighted and weighted graphs respectively with round complexity $\widetilde{\mathcal{O}}(\sqrt{n})$ in $\hybrid$, closely matching the performance of the state of the art randomized algorithm of Kuhn and Schneider \cite{10.1145/3382734.3405719}. Moreover, we make a connection with the Ghaffari-Haeupler framework of low-congestion shortcuts \cite{DBLP:conf/soda/GhaffariH16}, leading -- among others -- to a $(1 + ε)$-approximate algorithm for Min-Cut after $\log^{\mathcal{O}(1)}n$ rounds, with high probability, even if we restrict local edges to transfer $\mathcal{O}(\log n)$-bits per round. Finally, we prove via a reduction from the set disjointness problem that $\widetildeΩ(n^{1/3})$ rounds are required to determine the radius of an unweighted graph, as well as a $(3/2 - ε)$-approximation for weighted graphs.
Pandemic and disability: Challenges faced and role of technology
Venue: arXiv cs.HC 2021
Year: 2021

The pandemic has affected every facet of human life. Apart from individuals psychological and mental health issues, the concern regarding mobility, access and communication with high risk infection is a challenging situation. People with disability are more likely vulnerable to infections. The new changes in our social lifestyle (social distancing, limiting touch) can profoundly impact the day today life of people with disability. In this paper, we will briefly discuss the situation faced by individuals with disabilities, some known remedies, and yet to be identified and curated technological remedies; the impact due to transition of special education toward online mode. Tips and tricks for better utilization of work from home concept by people with disabilities. Accessibility must be universal, accommodating all and encouraging inclusivity. As rightly said by Helen Keller, 'The only thing worse than being blind is having sight but no vision'; subsequently, going by the demand of the time, we should contribute toward the universal design approach by supporting people with disabilities and commit to the changes required in disability care to reduce the impact of pandemic. Keywords: Disabilities, pandemic, corona virus, inclusive
Finite Horizon Privacy of Stochastic Dynamical Systems: A Synthesis Framework for Dependent Gaussian Mechanisms
Venue: arXiv cs.CR 2021
Year: 2021

We address the problem of synthesizing distorting mechanisms that maximize privacy of stochastic dynamical systems. Information about the system state is obtained through sensor measurements. This data is transmitted to a remote station through an unsecured/public communication network. We aim to keep part of the system state private (a private output); however, because the network is unsecured, adversaries might access sensor data and input signals, which can be used to estimate private outputs. To prevent an accurate estimation, we pass sensor data and input signals through a distorting (privacy-preserving) mechanism before transmission, and send the distorted data to the trusted user. These mechanisms consist of a coordinate transformation and additive dependent Gaussian vectors. We formulate the synthesis of the distorting mechanisms as a convex program, where we minimize the mutual information (our privacy metric) between an arbitrarily large sequence of private outputs and the disclosed distorted data for desired distortion levels -- how different actual and distorted data are allowed to be.
Armour: Generalizable Compact Self-Attention for Vision Transformers
Venue: arXiv cs.CV 2021
Year: 2021

Attention-based transformer networks have demonstrated promising potential as their applications extend from natural language processing to vision. However, despite the recent improvements, such as sub-quadratic attention approximation and various training enhancements, the compact vision transformers to date using the regular attention still fall short in comparison with its convnet counterparts, in terms of \textit{accuracy,} \textit{model size}, \textit{and} \textit{throughput}. This paper introduces a compact self-attention mechanism that is fundamental and highly generalizable. The proposed method reduces redundancy and improves efficiency on top of the existing attention optimizations. We show its drop-in applicability for both the regular attention mechanism and some most recent variants in vision transformers. As a result, we produced smaller and faster models with the same or better accuracies.
Leaf Recognition Using Convolutional Neural Networks Based Features
Venue: arXiv cs.CV 2021
Year: 2021

There is a warning light for the loss of plant habitats worldwide that entails concerted efforts to conserve plant biodiversity. Thus, plant species classification is of crucial importance to address this environmental challenge. In recent years, there is a considerable increase in the number of studies related to plant taxonomy. While some researchers try to improve their recognition performance using novel approaches, others concentrate on computational optimization for their framework. In addition, a few studies are diving into feature extraction to gain significantly in terms of accuracy. In this paper, we propose an effective method for the leaf recognition problem. In our proposed approach, a leaf goes through some pre-processing to extract its refined color image, vein image, xy-projection histogram, handcrafted shape, texture features, and Fourier descriptors. These attributes are then transformed into a better representation by neural network-based encoders before a support vector machine (SVM) model is utilized to classify different leaves. Overall, our approach performs a state-of-the-art result on the Flavia leaf dataset, achieving the accuracy of 99.58\% on test sets under random 10-fold cross-validation and bypassing the previous methods. We also release our codes (Scripts are available at https://github.com/dinhvietcuong1996/LeafRecognition) for contributing to the research community in the leaf classification problem.
Improving Distinction between ASR Errors and Speech Disfluencies with Feature Space Interpolation
Venue: arXiv cs.CL 2021
Year: 2021

Fine-tuning pretrained language models (LMs) is a popular approach to automatic speech recognition (ASR) error detection during post-processing. While error detection systems often take advantage of statistical language archetypes captured by LMs, at times the pretrained knowledge can hinder error detection performance. For instance, presence of speech disfluencies might confuse the post-processing system into tagging disfluent but accurate transcriptions as ASR errors. Such confusion occurs because both error detection and disfluency detection tasks attempt to identify tokens at statistically unlikely positions. This paper proposes a scheme to improve existing LM-based ASR error detection systems, both in terms of detection scores and resilience to such distracting auxiliary tasks. Our approach adopts the popular mixup method in text feature space and can be utilized with any black-box ASR output. To demonstrate the effectiveness of our method, we conduct post-processing experiments with both traditional and end-to-end ASR systems (both for English and Korean languages) with 5 different speech corpora. We find that our method improves both ASR error detection F 1 scores and reduces the number of correctly transcribed disfluencies wrongly detected as ASR errors. Finally, we suggest methods to utilize resulting LMs directly in semi-supervised ASR training.
Reconstructing a dynamical system and forecasting time series by self-consistent deep learning
Venue: arXiv cs.LG 2021
Year: 2021

We introduce a self-consistent deep-learning framework which, for a noisy deterministic time series, provides unsupervised filtering, state-space reconstruction, identification of the underlying differential equations and forecasting. Without a priori information on the signal, we embed the time series in a state space, where deterministic structures, i.e. attractors, are revealed. Under the assumption that the evolution of solution trajectories is described by an unknown dynamical system, we filter out stochastic outliers. The embedding function, the solution trajectories and the dynamical systems are constructed using deep neural networks, respectively. By exploiting the differentiability of the neural solution trajectory, the neural dynamical system is defined locally at each time, mitigating the need for propagating gradients through numerical solvers. On a chaotic time series masked by additive Gaussian noise, we demonstrate the filtering ability and the predictive power of the proposed framework.