OpenRaiser/Intern-Atlas huggingface dataset:
Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages
Venue: arXiv cs.CL 2022
Year: 2022

We introduce Wav2Seq, the first self-supervised approach to pre-train both parts of encoder-decoder models for speech data. We induce a pseudo language as a compact discrete representation, and formulate a self-supervised pseudo speech recognition task -- transcribing audio inputs into pseudo subword sequences. This process stands on its own, or can be applied as low-cost second-stage pre-training. We experiment with automatic speech recognition (ASR), spoken named entity recognition, and speech-to-text translation. We set new state-of-the-art results for end-to-end spoken named entity recognition, and show consistent improvements on 20 language pairs for speech-to-text translation, even when competing methods use additional text data for training. Finally, on ASR, our approach enables encoder-decoder methods to benefit from pre-training for all parts of the network, and shows comparable performance to highly optimized recent methods.
Leximax Approximations and Representative Cohort Selection
Venue: arXiv cs.DS 2022
Year: 2022

Finding a representative cohort from a broad pool of candidates is a goal that arises in many contexts such as choosing governing committees and consumer panels. While there are many ways to define the degree to which a cohort represents a population, a very appealing solution concept is lexicographic maximality (leximax) which offers a natural (pareto-optimal like) interpretation that the utility of no population can be increased without decreasing the utility of a population that is already worse off. However, finding a leximax solution can be highly dependent on small variations in the utility of certain groups. In this work, we explore new notions of approximate leximax solutions with three distinct motivations: better algorithmic efficiency, exploiting significant utility improvements, and robustness to noise. Among other definitional contributions, we give a new notion of an approximate leximax that satisfies a similarly appealing semantic interpretation and relate it to algorithmically-feasible approximate leximax notions. When group utilities are linear over cohort candidates, we give an efficient polynomial-time algorithm for finding a leximax distribution over cohort candidates in the exact as well as in the approximate setting. Furthermore, we show that finding an integer solution to leximax cohort selection with linear utilities is NP-Hard.
NHA12D: A New Pavement Crack Dataset and a Comparison Study Of Crack Detection Algorithms
Venue: arXiv cs.CV 2022
Year: 2022

Crack detection plays a key role in automated pavement inspection. Although a large number of algorithms have been developed in recent years to further boost performance, there are still remaining challenges in practice, due to the complexity of pavement images. To further accelerate the development and identify the remaining challenges, this paper conducts a comparison study to evaluate the performance of the state of the art crack detection algorithms quantitatively and objectively. A more comprehensive annotated pavement crack dataset (NHA12D) that contains images with different viewpoints and pavements types is proposed. In the comparison study, crack detection algorithms were trained equally on the largest public crack dataset collected and evaluated on the proposed dataset (NHA12D). Overall, the U-Net model with VGG-16 as backbone has the best all-around performance, but models generally fail to distinguish cracks from concrete joints, leading to a high false-positive rate. It also found that detecting cracks from concrete pavement images still has huge room for improvement. Dataset for concrete pavement images is also missing in the literature. Future directions in this area include filling the gap for concrete pavement images and using domain adaptation techniques to enhance the detection results on unseen datasets.
Multi-Task Text Classification using Graph Convolutional Networks for Large-Scale Low Resource Language
Venue: arXiv cs.CL 2022
Year: 2022

Graph Convolutional Networks (GCN) have achieved state-of-art results on single text classification tasks like sentiment analysis, emotion detection, etc. However, the performance is achieved by testing and reporting on resource-rich languages like English. Applying GCN for multi-task text classification is an unexplored area. Moreover, training a GCN or adopting an English GCN for Indian languages is often limited by data availability, rich morphological variation, syntax, and semantic differences. In this paper, we study the use of GCN for the Telugu language in single and multi-task settings for four natural language processing (NLP) tasks, viz. sentiment analysis (SA), emotion identification (EI), hate-speech (HS), and sarcasm detection (SAR). In order to evaluate the performance of GCN with one of the Indian languages, Telugu, we analyze the GCN based models with extensive experiments on four downstream tasks. In addition, we created an annotated Telugu dataset, TEL-NLP, for the four NLP tasks. Further, we propose a supervised graph reconstruction method, Multi-Task Text GCN (MT-Text GCN) on the Telugu that leverages to simultaneously (i) learn the low-dimensional word and sentence graph embeddings from word-sentence graph reconstruction using graph autoencoder (GAE) and (ii) perform multi-task text classification using these latent sentence graph embeddings. We argue that our proposed MT-Text GCN achieves significant improvements on TEL-NLP over existing Telugu pretrained word embeddings, and multilingual pretrained Transformer models: mBERT, and XLM-R. On TEL-NLP, we achieve a high F1-score for four NLP tasks: SA (0.84), EI (0.55), HS (0.83) and SAR (0.66). Finally, we show our model's quantitative and qualitative analysis on the four NLP tasks in Telugu.
Applications of Deep Learning to the Design of Enhanced Wireless Communication Systems
Venue: arXiv cs.IT 2022
Year: 2022

Innovation in the physical layer of communication systems has traditionally been achieved by breaking down the transceivers into sets of processing blocks, each optimized independently based on mathematical models. Conversely, deep learning (DL)-based systems are able to handle increasingly complex tasks for which no tractable models are available. This thesis aims at comparing different approaches to unlock the full potential of DL in the physical layer. First, we describe a neural network (NN)-based block strategy, where an NN is optimized to replace a block in a communication system. We apply this strategy to introduce a multi-user multiple-input multiple-output (MU-MIMO) detector that builds on top of an existing DL-based architecture. Second, we detail an end-to-end strategy, in which the transmitter and receiver are modeled as an autoencoder. This approach is illustrated with the design of waveforms that achieve high throughputs while satisfying peak-to-average power ratio (PAPR) and adjacent channel leakage ratio (ACLR) constraints. Lastly, we propose a hybrid strategy, where multiple DL components are inserted into a traditional architecture but are trained to optimize the end-to-end performance. To demonstrate its benefits, we propose a DL-enhanced MU-MIMO receiver that both enable lower bit error rates (BERs) compared to a conventional receiver and remains scalable to any number of users. Each approach has its own strengths and shortcomings. While the first one is the easiest to implement, its individual block optimization does not ensure the overall system optimality. On the other hand, systems designed with the second approach are computationally complex but allow for new opportunities such as pilotless transmissions. Finally, the combined flexibility and end-to-end performance gains of the third approach motivate its use for short-term practical implementations.
Line-of-Sight MIMO via Reflection From a Smooth Surface
Venue: arXiv cs.IT 2022
Year: 2022

We provide a deterministic channel model for a scenario where wireless connectivity is established through a reflection from a planar smooth surface of an infinite extent. The developed model is rigorously built upon the physics of wave propagation, and is as precise as tight are the unboundedness and smoothness assumptions on the surface. This model allows establishing that line-of-sight spatial multiplexing can take place via reflection off an electrically large surface, a situation of high interest for mmWave and terahertz frequencies.
Adversarial attacks on an optical neural network
Venue: arXiv cs.CR 2022
Year: 2022

Adversarial attacks have been extensively investigated for machine learning systems including deep learning in the digital domain. However, the adversarial attacks on optical neural networks (ONN) have been seldom considered previously. In this work, we first construct an accurate image classifier with an ONN using a mesh of interconnected Mach-Zehnder interferometers (MZI). Then a corresponding adversarial attack scheme is proposed for the first time. The attacked images are visually very similar to the original ones but the ONN system becomes malfunctioned and generates wrong classification results in most time. The results indicate that adversarial attack is also a significant issue for optical machine learning systems.
ADDAI: Anomaly Detection using Distributed AI
Venue: arXiv cs.DC 2022
Year: 2022

When dealing with the Internet of Things (IoT), especially industrial IoT (IIoT), two manifest challenges leap to mind. First is the massive amount of data streaming to and from IoT devices, and second is the fast pace at which these systems must operate. Distributed computing in the form of edge/cloud structure is a popular technique to overcome these two challenges. In this paper, we propose ADDAI (Anomaly Detection using Distributed AI) that can easily span out geographically to cover a large number of IoT sources. Due to its distributed nature, it guarantees critical IIoT requirements such as high speed, robustness against a single point of failure, low communication overhead, privacy, and scalability. Through empirical proof, we show the communication cost is minimized, and the performance improves significantly while maintaining the privacy of raw data at the local layer. ADDAI provides predictions for new random samples with an average success rate of 98.4% while reducing the communication overhead by half compared with the traditional technique of offloading all the raw sensor data to the cloud.
Meaningful Context, a Red Flag, or Both? Users' Preferences for Enhanced Misinformation Warnings on Twitter
Venue: arXiv cs.CY 2022
Year: 2022

Warning users about misinformation on social media is not a simple usability task. Soft moderation has to balance between debunking falsehoods and avoiding moderation bias while preserving the social media consumption flow. Platforms thus employ minimally distinguishable warning tags with generic text under a suspected misinformation content. This approach resulted in an unfavorable outcome where the warnings "backfired" and users believed the misinformation more, not less. In response, we developed enhancements to the misinformation warnings where users are advised on the context of the information hazard and exposed to standard warning iconography. We ran an A/B evaluation with the Twitter's original warning tags in a 337 participant usability study. The majority of the participants preferred the enhancements as a nudge toward recognizing and avoiding misinformation. The enhanced warning tags were most favored by the politically left-leaning and to a lesser degree moderate participants, but they also appealed to roughly a third of the right-leaning participants. The education level was the only demographic factor shaping participants' preferences. We use our findings to propose user-tailored improvements in the soft moderation of misinformation on social media.
Scheduling with Speed Predictions
Venue: arXiv cs.DS 2022
Year: 2022

Algorithms with predictions is a recent framework that has been used to overcome pessimistic worst-case bounds in incomplete information settings. In the context of scheduling, very recent work has leveraged machine-learned predictions to design algorithms that achieve improved approximation ratios in settings where the processing times of the jobs are initially unknown. In this paper, we study the speed-robust scheduling problem where the speeds of the machines, instead of the processing times of the jobs, are unknown and augment this problem with predictions. Our main result is an algorithm that achieves a $\min\{η^2(1+α), (2 + 2/α)\}$ approximation, for any $α\in (0,1)$, where $η\geq 1$ is the prediction error. When the predictions are accurate, this approximation outperforms the best known approximation for speed-robust scheduling without predictions of $2-1/m$, where $m$ is the number of machines, while simultaneously maintaining a worst-case approximation of $2 + 2/α$ even when the predictions are arbitrarily wrong. In addition, we obtain improved approximations for three special cases: equal job sizes, infinitesimal job sizes, and binary machine speeds. We also complement our algorithmic results with lower bounds. Finally, we empirically evaluate our algorithm against existing algorithms for speed-robust scheduling.
SIMD$^2$: A Generalized Matrix Instruction Set for Accelerating Tensor Computation beyond GEMM
Venue: arXiv cs.AR 2022
Year: 2022

Matrix-multiplication units (MXUs) are now prevalent in every computing platform. The key attribute that makes MXUs so successful is the semiring structure, which allows tiling for both parallelism and data reuse. Nonetheless, matrix-multiplication is not the only algorithm with such attributes. We find that many algorithms share the same structure and differ in only the core operation; for example, using add-minimum instead of multiply-add. Algorithms with a semiring-like structure therefore have potential to be accelerated by a general-purpose matrix operation architecture, instead of common MXUs. In this paper, we propose SIMD$^2$, a new programming paradigm to support generalized matrix operations with a semiring-like structure. SIMD$^2$ instructions accelerate eight more types of matrix operations, in addition to matrix multiplications. Since SIMD$^2$ instructions resemble a matrix-multiplication instruction, we are able to build SIMD$^2$ architecture on top of any MXU architecture with minimal modifications. We developed a framework that emulates and validates SIMD$^2$ using NVIDIA GPUs with Tensor Cores. Across 8 applications, SIMD2 provides up to 38.59$\times$ speedup and more than 10.63$\times$ on average over optimized CUDA programs, with only 5% of full-chip area overhead.
Storyteller: The papers co-citing Sleeping Beauty and Prince before awakening
Venue: arXiv cs.DL 2022
Year: 2022

In the Cumulative Advantage(CA) model, which is one of the most fundamental approaches to understand the mechanism of citation dynamics, papers receive citations depending on how much they have been already cited. On the other hand, a substantial effect not included in CA is that some surprising discoveries suddenly acquire citations after a long time from publishing. This phenomenon is known as Sleeping Beauty(SB). Since disrupting discoveries need long-time discussion by the research community to accept, SBs can capture innovative findings and reveal the nature of disruptive scientific knowledge production. To research SBs citation burst mechanism, bibliometricians consider the existence of the Prince(PR) for each SBs, which can be the trigger of SBs awakeness. For example, the discovery of Green Fluorescent Protein(GFP), which got Nobel prize in chemistry, had been overlooked for 30 years until Chalfie and Tsien, who also received the prize, developed a method to use GFP as a marker protein in genetic engineering. However, how does Chalfies and Tsiens research relight the hidden knowledge in the research community? If we can clarify such a mechanism rediscovering from nearly nothing, it can be helpful in science support and policy decision-making. This study proposes a Storyteller that focuses on the connection between SB and PR before SB gets citation burst by co-citation. PR is found to be the paper awakening SB in retrospect, but it is not easy to detect it as the trigger of SBs awakeness at the time of PR submission. We named the papers which co-cites SB and PR before the citation burst of SB as Storyteller(ST) and analyze (1) how ST contributes to broadening the novelty of SB&PR connections and (2) how much ST leads the citation burst after awakening.
Engineering Annotations: A Generic Framework For Gluing Design Artefacts in Models of Interactive Systems
Venue: arXiv cs.HC 2022
Year: 2022

Along the design process of interactive system many intermediate artefacts (such as user interface prototypes, task models describing user work and activities, dialog models specifying system behavior, interaction models describing user interactions {\ldots}) are created, tested, revised and improved until the development team produces a validated version of the full-fledged system. Indeed, to build interactive systems there is a need to use multiple artefacts/models (as they provide a complementary view). However, relevant information for describing the design solution and/or supporting design decisions (such as rational about the design, decisions made, recommendations, etc.) is not explicitly capturable in the models/artefacts, hence the need for annotations. Multi-artefacts approaches usually argue that a given information should only be present in one artefact to avoid duplication and increase maintainability of the artefacts. Nonetheless, annotations created on one artefact are usually relevant to other artefacts/models. So that, there is a need for tools and techniques to coordinate annotations across artefacts/models which is the contribution of the present work. In this paper, we propose a model-based approach that was conceived to handle annotations in a systematic way along the development process of interactive systems. As part of the solution, we propose an annotation model built upon the W3C's Web Annotation Data Model. The feasibility of the approach is demonstrated by means of a tool suite featuring a plugin, which has been deployed and tested over the multi-artefacts. The overall approach is illustrated on the design of an interactive cockpit application performing two design iterations. The contribution brings two main benefits for interactive systems engineering: i) it presents a generic pattern for integrating information in multiple usually heterogenous artefacts throughout the design process of interactive systems; and ii) it highlights the need for tools helping to rationalize and to document the various artefacts and the related decisions made during interactive systems design. CCS CONCEPTS $\bullet$ Human-centered computing $\bullet$ Human computer interaction (HCI)
BioTouchPass: Handwritten Passwords for Touchscreen Biometrics
Venue: arXiv cs.CR 2022
Year: 2022

This work enhances traditional authentication systems based on Personal Identification Numbers (PIN) and One-Time Passwords (OTP) through the incorporation of biometric information as a second level of user authentication. In our proposed approach, users draw each digit of the password on the touchscreen of the device instead of typing them as usual. A complete analysis of our proposed biometric system is carried out regarding the discriminative power of each handwritten digit and the robustness when increasing the length of the password and the number of enrolment samples. The new e-BioDigit database, which comprises on-line handwritten digits from 0 to 9, has been acquired using the finger as input on a mobile device. This database is used in the experiments reported in this work and it is available together with benchmark results in GitHub. Finally, we discuss specific details for the deployment of our proposed approach on current PIN and OTP systems, achieving results with Equal Error Rates (EERs) ca. 4.0% when the attacker knows the password. These results encourage the deployment of our proposed approach in comparison to traditional PIN and OTP systems where the attack would have 100% success rate under the same impostor scenario.
A Model of Random Multiple Access in Unlicensed Spectrum Systems
Venue: arXiv cs.IT 2022
Year: 2022

We consider a classical multiple access system with a single transmission channel, finite number of users (users), and randomized transmission protocol (ALOHA). We assume that every user sends messages to the base station with various intensities. Due to the overlapping of messages during their sending there are restrictions on the time between the messages and a mathematical model adequate to the physical one becomes quite complicated. In this work, we propose a simplified mathematical model that is easy to analyze and takes into account the properties of real systems.
RLFlow: Optimising Neural Network Subgraph Transformation with World Models
Venue: arXiv cs.LG 2022
Year: 2022

Training deep learning models takes an extremely long execution time and consumes large amounts of computing resources. At the same time, recent research proposed systems and compilers that are expected to decrease deep learning models runtime. An effective optimisation methodology in data processing is desirable, and the reduction of compute requirements of deep learning models is the focus of extensive research. In this paper, we address the neural network sub-graph transformation by exploring reinforcement learning (RL) agents to achieve performance improvement. Our proposed approach RLFlow can learn to perform neural network subgraph transformations, without the need for expertly designed heuristics to achieve a high level of performance. Recent work has aimed at applying RL to computer systems with some success, especially using model-free RL techniques. Model-based reinforcement learning methods have seen an increased focus in research as they can be used to learn the transition dynamics of the environment; this can be leveraged to train an agent using a hallucinogenic environment such as World Model (WM), thereby increasing sample efficiency compared to model-free approaches. WM uses variational auto-encoders and it builds a model of the system and allows exploring the model in an inexpensive way. In RLFlow, we propose a design for a model-based agent with WM which learns to optimise the architecture of neural networks by performing a sequence of sub-graph transformations to reduce model runtime. We show that our approach can match the state-of-the-art performance on common convolutional networks and outperforms by up to 5% those based on transformer-style architectures
Inducing and Using Alignments for Transition-based AMR Parsing
Venue: arXiv cs.CL 2022
Year: 2022

Transition-based parsers for Abstract Meaning Representation (AMR) rely on node-to-word alignments. These alignments are learned separately from parser training and require a complex pipeline of rule-based components, pre-processing, and post-processing to satisfy domain-specific constraints. Parsers also train on a point-estimate of the alignment pipeline, neglecting the uncertainty due to the inherent ambiguity of alignment. In this work we explore two avenues for overcoming these limitations. First, we propose a neural aligner for AMR that learns node-to-word alignments without relying on complex pipelines. We subsequently explore a tighter integration of aligner and parser training by considering a distribution over oracle action sequences arising from aligner uncertainty. Empirical results show this approach leads to more accurate alignments and generalization better from the AMR2.0 to AMR3.0 corpora. We attain a new state-of-the art for gold-only trained models, matching silver-trained performance without the need for beam search on AMR3.0.