OpenRaiser/Intern-Atlas huggingface dataset:
Coalition Formation Games for Improved Cell-Edge User Service in Downlink NOMA and MU-MIMO Small Cell Systems
Venue: arXiv cs.IT 2021
Year: 2021

In today's wireless communication systems, the exponentially growing needs of mobile users require the combination of new and existing techniques to meet the demands for reliable and high-quality service provision. This may not always be possible, as the resources in wireless telecommunication systems are limited and a significant number of users, usually located at the cell edges, can suffer from severe interference. For this purpose, a new Joint Transmission Coordinated Multipoint (JT-CoMP) scheme, in which the transmission points' clustering is based on a coalition formation game, is deployed alongside with Non-Orthogonal Multiple Access (NOMA) in a Cloud Radio Access Network (C-RAN) consisting of small cells. To further enhance the network's performance, Multiuser Multiple-Input Multiple-Output (MU-MIMO) with Zero-Forcing (ZF) beamforming is applied. The proposed scheme's performance, in terms of user throughput, is then compared to that of a scheme where JT-CoMP with static clustering is selected, a JT-CoMP scheme with clustering based on a greedy algorithm and a scenario where JT-CoMP is not deployed. Simulation results confirm that the proposed scheme reliably improves the throughput of users with poor wireless channels while guaranteeing that the performance of the rest is not severely undermined.
Sounds of COVID-19: exploring realistic performance of audio-based digital testing
Venue: arXiv cs.SD 2021
Year: 2021

Researchers have been battling with the question of how we can identify Coronavirus disease (COVID-19) cases efficiently, affordably and at scale. Recent work has shown how audio based approaches, which collect respiratory audio data (cough, breathing and voice) can be used for testing, however there is a lack of exploration of how biases and methodological decisions impact these tools' performance in practice. In this paper, we explore the realistic performance of audio-based digital testing of COVID-19. To investigate this, we collected a large crowdsourced respiratory audio dataset through a mobile app, alongside recent COVID-19 test result and symptoms intended as a ground truth. Within the collected dataset, we selected 5,240 samples from 2,478 participants and split them into different participant-independent sets for model development and validation. Among these, we controlled for potential confounding factors (such as demographics and language). The unbiased model takes features extracted from breathing, coughs, and voice signals as predictors and yields an AUC-ROC of 0.71 (95\% CI: 0.65$-$0.77). We further explore different unbalanced distributions to show how biases and participant splits affect performance. Finally, we discuss how the realistic model presented could be integrated in clinical practice to realize continuous, ubiquitous, sustainable and affordable testing at population scale.
Generalizing Syndrome Decoding problem to the totally Non-negative Grassmannian
Venue: arXiv cs.IT 2021
Year: 2021

The syndrome decoding problem has been proposed as a computational hardness assumption for code based cryptosystem that are safe against quantum computing. The problem has been reduced to finding the codeword with the smallest non-zero columns that would satisfy a linear check equation. Variants of Information set decoding algorithms has been developed as cryptanalytic tools to solve the problem. In this paper, we study and generalize the solution to codes associated with the totally non-negative Grassmannian in the Grassmann metric. This is achieved by reducing it to an instance of finding a subset of the plucker coordinates with the smallest number of columns. Subsequently, the theory of the totally non negative Grassmann is extended to connect the concept of boundary measurement map to Tanner graph like code construction while deriving new analytical bounds on its parameters. The derived bounds shows that the complexity scales up on the size of the plucker coordinates. Finally, experimental results on decoding failure probability and complexity based on row operations are presented and compared to Low Density parity check codes in the Hamming metric.
Hate speech detection using static BERT embeddings
Venue: arXiv cs.CL 2021
Year: 2021

With increasing popularity of social media platforms hate speech is emerging as a major concern, where it expresses abusive speech that targets specific group characteristics, such as gender, religion or ethnicity to spread violence. Earlier people use to verbally deliver hate speeches but now with the expansion of technology, some people are deliberately using social media platforms to spread hate by posting, sharing, commenting, etc. Whether it is Christchurch mosque shootings or hate crimes against Asians in west, it has been observed that the convicts are very much influenced from hate text present online. Even though AI systems are in place to flag such text but one of the key challenges is to reduce the false positive rate (marking non hate as hate), so that these systems can detect hate speech without undermining the freedom of expression. In this paper, we use ETHOS hate speech detection dataset and analyze the performance of hate speech detection classifier by replacing or integrating the word embeddings (fastText (FT), GloVe (GV) or FT + GV) with static BERT embeddings (BE). With the extensive experimental trails it is observed that the neural network performed better with static BE compared to using FT, GV or FT + GV as word embeddings. In comparison to fine-tuned BERT, one metric that significantly improved is specificity.
Evaluation of Automated Image Descriptions for Visually Impaired Students
Venue: arXiv cs.HC 2021
Year: 2021

Illustrations are widely used in education, and sometimes, alternatives are not available for visually impaired students. Therefore, those students would benefit greatly from an automatic illustration description system, but only if those descriptions were complete, correct, and easily understandable using a screenreader. In this paper, we report on a study for the assessment of automated image descriptions. We interviewed experts to establish evaluation criteria, which we then used to create an evaluation questionnaire for sighted non-expert raters, and description templates. We used this questionnaire to evaluate the quality of descriptions which could be generated with a template-based automatic image describer. We present evidence that these templates have the potential to generate useful descriptions, and that the questionnaire identifies problems with description templates.
Flare: Flexible In-Network Allreduce
Venue: arXiv cs.DC 2021
Year: 2021

The allreduce operation is one of the most commonly used communication routines in distributed applications. To improve its bandwidth and to reduce network traffic, this operation can be accelerated by offloading it to network switches, that aggregate the data received from the hosts, and send them back the aggregated result. However, existing solutions provide limited customization opportunities and might provide suboptimal performance when dealing with custom operators and data types, with sparse data, or when reproducibility of the aggregation is a concern. To deal with these problems, in this work we design a flexible programmable switch by using as a building block PsPIN, a RISC-V architecture implementing the sPIN programming model. We then design, model, and analyze different algorithms for executing the aggregation on this architecture, showing performance improvements compared to state-of-the-art approaches.
Segmentation with Multiple Acceptable Annotations: A Case Study of Myocardial Segmentation in Contrast Echocardiography
Venue: arXiv cs.CV 2021
Year: 2021

Most existing deep learning-based frameworks for image segmentation assume that a unique ground truth is known and can be used for performance evaluation. This is true for many applications, but not all. Myocardial segmentation of Myocardial Contrast Echocardiography (MCE), a critical task in automatic myocardial perfusion analysis, is an example. Due to the low resolution and serious artifacts in MCE data, annotations from different cardiologists can vary significantly, and it is hard to tell which one is the best. In this case, how can we find a good way to evaluate segmentation performance and how do we train the neural network? In this paper, we address the first problem by proposing a new extended Dice to effectively evaluate the segmentation performance when multiple accepted ground truth is available. Then based on our proposed metric, we solve the second problem by further incorporating the new metric into a loss function that enables neural networks to flexibly learn general features of myocardium. Experiment results on our clinical MCE data set demonstrate that the neural network trained with the proposed loss function outperforms those existing ones that try to obtain a unique ground truth from multiple annotations, both quantitatively and qualitatively. Finally, our grading study shows that using extended Dice as an evaluation metric can better identify segmentation results that need manual correction compared with using Dice.
Diff2Dist: Learning Spectrally Distinct Edge Functions, with Applications to Cell Morphology Analysis
Venue: arXiv cs.LG 2021
Year: 2021

We present a method for learning "spectrally descriptive" edge weights for graphs. We generalize a previously known distance measure on graphs (Graph Diffusion Distance), thereby allowing it to be tuned to minimize an arbitrary loss function. Because all steps involved in calculating this modified GDD are differentiable, we demonstrate that it is possible for a small neural network model to learn edge weights which minimize loss. GDD alone does not effectively discriminate between graphs constructed from shoot apical meristem images of wild-type vs. mutant \emph{Arabidopsis thaliana} specimens. However, training edge weights and kernel parameters with contrastive loss produces a learned distance metric with large margins between these graph categories. We demonstrate this by showing improved performance of a simple k-nearest-neighbors classifier on the learned distance matrix. We also demonstrate a further application of this method to biological image analysis: once trained, we use our model to compute the distance between the biological graphs and a set of graphs output by a cell division simulator. This allows us to identify simulation parameter regimes which are similar to each class of graph in our original dataset.
($S$,$N$,$T$)-Implications
Venue: arXiv cs.LO 2021
Year: 2021

In this paper we introduce a new class of fuzzy implications called ($S$,$N$,$T$)-implications inspired in the logical equivalence $p\rightarrow q \equiv \neg(p\wedge\neg q)\vee\neg p$ and present a brief study of some of the main properties that characterize this class. We present methods of obtaining $t$-norms and $t$-conorms from an ($S$,$N$,$T$)-implication and a fuzzy negation.
Dual GNNs: Graph Neural Network Learning with Limited Supervision
Venue: arXiv cs.LG 2021
Year: 2021

Graph Neural Networks (GNNs) require a relatively large number of labeled nodes and a reliable/uncorrupted graph connectivity structure in order to obtain good performance on the semi-supervised node classification task. The performance of GNNs can degrade significantly as the number of labeled nodes decreases or the graph connectivity structure is corrupted by adversarial attacks or due to noises in data measurement /collection. Therefore, it is important to develop GNN models that are able to achieve good performance when there is limited supervision knowledge -- a few labeled nodes and noisy graph structures. In this paper, we propose a novel Dual GNN learning framework to address this challenge task. The proposed framework has two GNN based node prediction modules. The primary module uses the input graph structure to induce regular node embeddings and predictions with a regular GNN baseline, while the auxiliary module constructs a new graph structure through fine-grained spectral clusterings and learns new node embeddings and predictions. By integrating the two modules in a dual GNN learning framework, we perform joint learning in an end-to-end fashion. This general framework can be applied on many GNN baseline models. The experimental results validate that the proposed dual GNN framework can greatly outperform the GNN baseline methods when the labeled nodes are scarce and the graph connectivity structure is noisy.
A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers
Venue: arXiv cs.AI 2021
Year: 2021

We present ASDiv (Academia Sinica Diverse MWP Dataset), a diverse (in terms of both language patterns and problem types) English math word problem (MWP) corpus for evaluating the capability of various MWP solvers. Existing MWP corpora for studying AI progress remain limited either in language usage patterns or in problem types. We thus present a new English MWP corpus with 2,305 MWPs that cover more text patterns and most problem types taught in elementary school. Each MWP is annotated with its problem type and grade level (for indicating the level of difficulty). Furthermore, we propose a metric to measure the lexicon usage diversity of a given MWP corpus, and demonstrate that ASDiv is more diverse than existing corpora. Experiments show that our proposed corpus reflects the true capability of MWP solvers more faithfully.
Semantic Segmentation of Periocular Near-Infra-Red Eye Images Under Alcohol Effects
Venue: arXiv cs.CV 2021
Year: 2021

This paper proposes a new framework to detect, segment, and estimate the localization of the eyes from a periocular Near-Infra-Red iris image under alcohol consumption. The purpose of the system is to measure the fitness for duty. Fitness systems allow us to determine whether a person is physically or psychologically able to perform their tasks. Our framework is based on an object detector trained from scratch to detect both eyes from a single image. Then, two efficient networks were used for semantic segmentation; a Criss-Cross attention network and DenseNet10, with only 122,514 and 210,732 parameters, respectively. These networks can find the pupil, iris, and sclera. In the end, the binary output eye mask is used for pupil and iris diameter estimation with high precision. Five state-of-the-art algorithms were used for this purpose. A mixed proposal reached the best results. A second contribution is establishing an alcohol behavior curve to detect the alcohol presence utilizing a stream of images captured from an iris instance. Also, a manually labeled database with more than 20k images was created. Our best method obtains a mean Intersection-over-Union of 94.54% with DenseNet10 with only 210,732 parameters and an error of only 1-pixel on average.
Optimal Construction for Time-Convex Hull with Two Orthogonal Highways in the L1-metric
Venue: arXiv cs.CG 2021
Year: 2021

We consider the time-convex hull problem in the presence of two orthogonal highways H. In this problem, the travelling speed on the highway is faster than off the highway, and the time-convex hull of a point set P is the closure of P with respect to the inclusion of shortest time-paths. In this paper, we provide the algorithm for constructing the time-convex hull with two orthogonal highways. We reach the optimal result of O(n log n) time for arbitrary highway speed in the L1-metric. For the L2-metric with infinite highway speed, we hit the goal of O(n log n) time as well.
Faithful Edge Federated Learning: Scalability and Privacy
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning enables machine learning algorithms to be trained over a network of multiple decentralized edge devices without requiring the exchange of local datasets. Successfully deploying federated learning requires ensuring that agents (e.g., mobile devices) faithfully execute the intended algorithm, which has been largely overlooked in the literature. In this study, we first use risk bounds to analyze how the key feature of federated learning, unbalanced and non-i.i.d. data, affects agents' incentives to voluntarily participate and obediently follow traditional federated learning algorithms. To be more specific, our analysis reveals that agents with less typical data distributions and relatively more samples are more likely to opt out of or tamper with federated learning algorithms. To this end, we formulate the first faithful implementation problem of federated learning and design two faithful federated learning mechanisms which satisfy economic properties, scalability, and privacy. Further, the time complexity of computing all agents' payments in the number of agents is $\mathcal{O}(1)$. First, we design a Faithful Federated Learning (FFL) mechanism which approximates the Vickrey-Clarke-Groves (VCG) payments via an incremental computation. We show that it achieves (probably approximate) optimality, faithful implementation, voluntary participation, and some other economic properties (such as budget balance). Second, by partitioning agents into several subsets, we present a scalable VCG mechanism approximation. We further design a scalable and Differentially Private FFL (DP-FFL) mechanism, the first differentially private faithful mechanism, that maintains the economic properties. Our mechanism enables one to make three-way performance tradeoffs among privacy, the iterations needed, and payment accuracy loss.
Informed Machine Learning for Improved Similarity Assessment in Process-Oriented Case-Based Reasoning
Venue: arXiv cs.AI 2021
Year: 2021

Currently, Deep Learning (DL) components within a Case-Based Reasoning (CBR) application often lack the comprehensive integration of available domain knowledge. The trend within machine learning towards so-called Informed machine learning can help to overcome this limitation. In this paper, we therefore investigate the potential of integrating domain knowledge into Graph Neural Networks (GNNs) that are used for similarity assessment between semantic graphs within process-oriented CBR applications. We integrate knowledge in two ways: First, a special data representation and processing method is used that encodes structural knowledge about the semantic annotations of each graph node and edge. Second, the message-passing component of the GNNs is constrained by knowledge on legal node mappings. The evaluation examines the quality and training time of the extended GNNs, compared to the stock models. The results show that both extensions are capable of providing better quality, shorter training times, or in some configurations both advantages at once.
The Impact of Disinformation on a Controversial Debate on Social Media
Venue: arXiv cs.SI 2021
Year: 2021

In this work we study how pervasive is the presence of disinformation in the Italian debate around immigration on Twitter and the role of automated accounts in the diffusion of such content. By characterising the Twitter users with an \textit{Untrustworthiness} score, that tells us how frequently they engage with disinformation content, we are able to see that such bad information consumption habits are not equally distributed across the users; adopting a network analysis approach, we can identify communities characterised by a very high presence of users that frequently share content from unreliable news sources. Within this context, social bots tend to inject in the network more malicious content, that often remains confined in a limited number of clusters; instead, they target reliable content in order to diversify their reach. The evidence we gather suggests that, at least in this particular case study, there is a strong interplay between social bots and users engaging with unreliable content, influencing the diffusion of the latter across the network.
Cross-lingual alignments of ELMo contextual embeddings
Venue: arXiv cs.CL 2021
Year: 2021

Building machine learning prediction models for a specific NLP task requires sufficient training data, which can be difficult to obtain for less-resourced languages. Cross-lingual embeddings map word embeddings from a less-resourced language to a resource-rich language so that a prediction model trained on data from the resource-rich language can also be used in the less-resourced language. To produce cross-lingual mappings of recent contextual embeddings, anchor points between the embedding spaces have to be words in the same context. We address this issue with a novel method for creating cross-lingual contextual alignment datasets. Based on that, we propose several cross-lingual mapping methods for ELMo embeddings. The proposed linear mapping methods use existing Vecmap and MUSE alignments on contextual ELMo embeddings. Novel nonlinear ELMoGAN mapping methods are based on GANs and do not assume isomorphic embedding spaces. We evaluate the proposed mapping methods on nine languages, using four downstream tasks: named entity recognition (NER), dependency parsing (DP), terminology alignment, and sentiment analysis. The ELMoGAN methods perform very well on the NER and terminology alignment tasks, with a lower cross-lingual loss for NER compared to the direct training on some languages. In DP and sentiment analysis, linear contextual alignment variants are more successful.
Distributed Nash Equilibrium Seeking under Quantization Communication
Venue: arXiv cs.DC 2021
Year: 2021

This paper investigates Nash equilibrium (NE) seeking problems for noncooperative games over multi-players networks with finite bandwidth communication. A distributed quantized algorithm is presented, which consists of local gradient play, distributed decision estimating, and adaptive quantization. Exponential convergence of the algorithm is established, and a relationship between the convergence rate and the bandwidth is quantitatively analyzed. Finally, a simulation of an energy consumption game is presented to validate the proposed results.
Improving the Efficiency of Transformers for Resource-Constrained Devices
Venue: arXiv cs.LG 2021
Year: 2021

Transformers provide promising accuracy and have become popular and used in various domains such as natural language processing and computer vision. However, due to their massive number of model parameters, memory and computation requirements, they are not suitable for resource-constrained low-power devices. Even with high-performance and specialized devices, the memory bandwidth can become a performance-limiting bottleneck. In this paper, we present a performance analysis of state-of-the-art vision transformers on several devices. We propose to reduce the overall memory footprint and memory transfers by clustering the model parameters. We show that by using only 64 clusters to represent model parameters, it is possible to reduce the data transfer from the main memory by more than 4x, achieve up to 22% speedup and 39% energy savings on mobile devices with less than 0.1% accuracy loss.