OpenRaiser/Intern-Atlas huggingface dataset:
Enriching Non-Autoregressive Transformer with Syntactic and SemanticStructures for Neural Machine Translation
Venue: arXiv cs.CL 2021
Year: 2021

The non-autoregressive models have boosted the efficiency of neural machine translation through parallelized decoding at the cost of effectiveness when comparing with the autoregressive counterparts. In this paper, we claim that the syntactic and semantic structures among natural language are critical for non-autoregressive machine translation and can further improve the performance. However, these structures are rarely considered in the existing non-autoregressive models. Inspired by this intuition, we propose to incorporate the explicit syntactic and semantic structures of languages into a non-autoregressive Transformer, for the task of neural machine translation. Moreover, we also consider the intermediate latent alignment within target sentences to better learn the long-term token dependencies. Experimental results on two real-world datasets (i.e., WMT14 En-De and WMT16 En-Ro) show that our model achieves a significantly faster speed, as well as keeps the translation quality when compared with several state-of-the-art non-autoregressive models.
PEQcheck: Localized and Context-aware Checking of Functional Equivalence (Technical Report)
Venue: arXiv cs.LO 2021
Year: 2021

Refactorings must not alter the program's functionality. However, not all refactorings fulfill this requirement. Hence, one must explicitly check that a refactoring does not alter the functionality. Since one rarely has a formal specification of the program's behavior, we utilize the original program as functional specification. Then, we check whether the original and refactored program are functionally equivalent. To this end, we apply a common idea and reduce equivalence checking to program verification. To increase efficiency, our equivalence checker PEQcheck constructs one verification task per refactored code segment instead of one per function as typically done by prior work. In addition, PEQcheck considers the context of the code segments. For instance, only variables that are modified and live are required to be equivalent and read-only variables may be shared between original and refactored code segments. We show that PEQcheck is sound.Moreover, our evaluation testifies that the localized and context-aware checking performed by \peqcheck can indeed be beneficial.
An ensemble solver for segregated cardiovascular FSI
Venue: arXiv cs.CE 2021
Year: 2021

Computational models are increasingly used for diagnosis and treatment of cardiovascular disease. To provide a quantitative hemodynamic understanding that can be effectively used in the clinic, it is crucial to quantify the variability in the outputs from these models due to multiple sources of uncertainty. To quantify this variability, the analyst invariably needs to generate a large collection of high-fidelity model solutions, typically requiring a substantial computational effort. In this paper, we show how an explicit-in-time ensemble cardiovascular solver offers superior performance with respect to the embarrassingly parallel solution with implicit-in-time algorithms, typical of an inner-outer loop paradigm for non-intrusive uncertainty propagation. We discuss in detail the numerics and efficient distributed implementation of a segregated FSI cardiovascular solver on both CPU and GPU systems, and demonstrate its applicability to idealized and patient-specific cardiovascular models, analyzed under steady and pulsatile flow conditions.
Deepfakes and the 2020 US elections: what (did not) happen
Venue: arXiv cs.SI 2021
Year: 2021

Alarmed by the volume of disinformation that was assumed to have taken place during the 2016 US elections, scholars, politics and journalists predicted the worst when the first deepfakes began to emerge in 2018. After all, US Elections 2020 were believed to be the most secure in American history. This paper seeks explanations for an apparent contradiction: we believe that it was precisely the multiplication and conjugation of different types of warnings and fears that created the conditions that prevented malicious political deepfakes from affecting the 2020 US elections. From these warnings, we identified four factors (more active role of social networks, new laws, difficulties in accessing Artificial Intelligence and better awareness of society). But while this formula has proven to be effective in the case of the United States, 2020, it is not correct to assume that it can be repeated in other political contexts.
A multi-perspective combined recall and rank framework for Chinese procedure terminology normalization
Venue: arXiv cs.CL 2021
Year: 2021

Medical terminology normalization aims to map the clinical mention to terminologies come from a knowledge base, which plays an important role in analyzing Electronic Health Record(EHR) and many downstream tasks. In this paper, we focus on Chinese procedure terminology normalization. The expression of terminologies are various and one medical mention may be linked to multiple terminologies. Previous study explores some methods such as multi-class classification or learning to rank(LTR) to sort the terminologies by literature and semantic information. However, these information is inadequate to find the right terminologies, particularly in multi-implication cases. In this work, we propose a combined recall and rank framework to solve the above problems. This framework is composed of a multi-task candidate generator(MTCG), a keywords attentive ranker(KAR) and a fusion block(FB). MTCG is utilized to predict the mention implication number and recall candidates with semantic similarity. KAR is based on Bert with a keywords attentive mechanism which focuses on keywords such as procedure sites and procedure types. FB merges the similarity come from MTCG and KAR to sort the terminologies from different perspectives. Detailed experimental analysis shows our proposed framework has a remarkable improvement on both performance and efficiency.
A Study of Continuous Vector Representationsfor Theorem Proving
Venue: arXiv cs.LG 2021
Year: 2021

Applying machine learning to mathematical terms and formulas requires a suitable representation of formulas that is adequate for AI methods. In this paper, we develop an encoding that allows for logical properties to be preserved and is additionally reversible. This means that the tree shape of a formula including all symbols can be reconstructed from the dense vector representation. We do that by training two decoders: one that extracts the top symbol of the tree and one that extracts embedding vectors of subtrees. The syntactic and semantic logical properties that we aim to reserve include both structural formula properties, applicability of natural deduction steps, and even more complex operations like unifiability. We propose datasets that can be used to train these syntactic and semantic properties. We evaluate the viability of the developed encoding across the proposed datasets as well as for the practical theorem proving problem of premise selection in the Mizar corpus.
The Next Decade of Telecommunications Artificial Intelligence
Venue: arXiv cs.NI 2021
Year: 2021

It has been an exciting journey since the mobile communications and artificial intelligence were conceived 37 years and 64 years ago. While both fields evolved independently and profoundly changed communications and computing industries, the rapid convergence of 5G and deep learning is beginning to significantly transform the core communication infrastructure, network management and vertical applications. The paper first outlines the individual roadmaps of mobile communications and artificial intelligence in the early stage, with a concentration to review the era from 3G to 5G when AI and mobile communications started to converge. With regard to telecommunications artificial intelligence, the paper further introduces in detail the progress of artificial intelligence in the ecosystem of mobile communications. The paper then summarizes the classifications of AI in telecom ecosystems along with its evolution paths specified by various international telecommunications standardization bodies. Towards the next decade, the paper forecasts the prospective roadmap of telecommunications artificial intelligence. In line with 3GPP and ITU-R timeline of 5G & 6G, the paper further explores the network intelligence following 3GPP and ORAN routes respectively, experience and intention driven network management and operation, network AI signalling system, intelligent middle-office based BSS, intelligent customer experience management and policy control driven by BSS and OSS convergence, evolution from SLA to ELA, and intelligent private network for verticals. The paper is concluded with the vision that AI will reshape the future B5G or 6G landscape and we need pivot our R&D, standardizations, and ecosystem to fully take the unprecedented opportunities.
Improved Sensitivity of Base Layer on the Performance of Rigid Pavement
Venue: arXiv cs.AI 2021
Year: 2021

The performance of rigid pavement is greatly affected by the properties of base/subbase as well as subgrade layer. However, the performance predicted by the AASHTOWare Pavement ME design shows low sensitivity to the properties of base and subgrade layers. To improve the sensitivity and better reflect the influence of unbound layers a new set of improved models i.e., resilient modulus (MR) and modulus of subgrade reaction (k-value) are adopted in this study. An Artificial Neural Network (ANN) model is developed to predict the modified k-value based on finite element (FE) analysis. The training and validation datasets in the ANN model consist of 27000 simulation cases with different combinations of pavement layer thickness, layer modulus and slab-base interface bond ratio. To examine the sensitivity of modified MR and k-values on pavement response, eight pavement sections data are collected from the Long-Term Pavement performance (LTPP) database and modeled by using the FE software ISLAB2000. The computational results indicate that the modified MR values have higher sensitivity to water content in base layer on critical stress and deflection response of rigid pavements compared to the results using the Pavement ME design model. It is also observed that the k-values using ANN model has the capability of predicting critical pavement response at any partially bonded conditions whereas the Pavement ME design model can only calculate at two extreme bonding conditions (i.e., fully bonding and no bonding).
Understanding Visual Saliency in Mobile User Interfaces
Venue: arXiv cs.HC 2021
Year: 2021

For graphical user interface (UI) design, it is important to understand what attracts visual attention. While previous work on saliency has focused on desktop and web-based UIs, mobile app UIs differ from these in several respects. We present findings from a controlled study with 30 participants and 193 mobile UIs. The results speak to a role of expectations in guiding where users look at. Strong bias toward the top-left corner of the display, text, and images was evident, while bottom-up features such as color or size affected saliency less. Classic, parameter-free saliency models showed a weak fit with the data, and data-driven models improved significantly when trained specifically on this dataset (e.g., NSS rose from 0.66 to 0.84). We also release the first annotated dataset for investigating visual saliency in mobile UIs.
Approximation capability of two hidden layer feedforward neural networks with fixed weights
Venue: arXiv cs.NE 2021
Year: 2021

We algorithmically construct a two hidden layer feedforward neural network (TLFN) model with the weights fixed as the unit coordinate vectors of the $d$-dimensional Euclidean space and having $3d+2$ number of hidden neurons in total, which can approximate any continuous $d$-variable function with an arbitrary precision. This result, in particular, shows an advantage of the TLFN model over the single hidden layer feedforward neural network (SLFN) model, since SLFNs with fixed weights do not have the capability of approximating multivariate functions.
Tensor-Train Networks for Learning Predictive Modeling of Multidimensional Data
Venue: arXiv cs.LG 2021
Year: 2021

In this work, we firstly apply the Train-Tensor (TT) networks to construct a compact representation of the classical Multilayer Perceptron, representing a reduction of up to 95% of the coefficients. A comparative analysis between tensor model and standard multilayer neural networks is also carried out in the context of prediction of the Mackey-Glass noisy chaotic time series and NASDAQ index. We show that the weights of a multidimensional regression model can be learned by means of TT network and the optimization of TT weights is a more robust to the impact of coefficient initialization and hyper-parameter setting. Furthermore, an efficient algorithm based on alternating least squares has been proposed for approximating the weights in TT-format with a reduction of computational calculus, providing a much faster convergence than the well-known adaptive learning-method algorithms, widely applied for optimizing neural networks.
It Takes Two to Tango: Combining Visual and Textual Information for Detecting Duplicate Video-Based Bug Reports
Venue: arXiv cs.SE 2021
Year: 2021

When a bug manifests in a user-facing application, it is likely to be exposed through the graphical user interface (GUI). Given the importance of visual information to the process of identifying and understanding such bugs, users are increasingly making use of screenshots and screen-recordings as a means to report issues to developers. However, when such information is reported en masse, such as during crowd-sourced testing, managing these artifacts can be a time-consuming process. As the reporting of screen-recordings in particular becomes more popular, developers are likely to face challenges related to manually identifying videos that depict duplicate bugs. Due to their graphical nature, screen-recordings present challenges for automated analysis that preclude the use of current duplicate bug report detection techniques. To overcome these challenges and aid developers in this task, this paper presents Tango, a duplicate detection technique that operates purely on video-based bug reports by leveraging both visual and textual information. Tango combines tailored computer vision techniques, optical character recognition, and text retrieval. We evaluated multiple configurations of Tango in a comprehensive empirical evaluation on 4,860 duplicate detection tasks that involved a total of 180 screen-recordings from six Android apps. Additionally, we conducted a user study investigating the effort required for developers to manually detect duplicate video-based bug reports and compared this to the effort required to use Tango. The results reveal that Tango's optimal configuration is highly effective at detecting duplicate video-based bug reports, accurately ranking target duplicate videos in the top-2 returned results in 83% of the tasks. Additionally, our user study shows that, on average, Tango can reduce developer effort by over 60%, illustrating its practicality.
Short Secret Sharing Using Repeatable Random Sequence Generators
Venue: arXiv cs.CR 2021
Year: 2021

We present a new secret sharing algorithm that provides the storage efficiency of an Information Dispersal Algorithm (IDA) while providing perfect secret sharing. We achieve this by mixing the input message with random bytes generated using Repeatable Random Sequence Generator (RRSG). We also use the data from the RRSG to provide random polynomial evaluation points and optionally compute the polynomials on random isomorphic fields rather than a single fixed field.
Next-best-view Regression using a 3D Convolutional Neural Network
Venue: arXiv cs.CV 2021
Year: 2021

Automated three-dimensional (3D) object reconstruction is the task of building a geometric representation of a physical object by means of sensing its surface. Even though new single view reconstruction techniques can predict the surface, they lead to incomplete models, specially, for non commons objects such as antique objects or art sculptures. Therefore, to achieve the task's goals, it is essential to automatically determine the locations where the sensor will be placed so that the surface will be completely observed. This problem is known as the next-best-view problem. In this paper, we propose a data-driven approach to address the problem. The proposed approach trains a 3D convolutional neural network (3D CNN) with previous reconstructions in order to regress the \btxt{position of the} next-best-view. To the best of our knowledge, this is one of the first works that directly infers the next-best-view in a continuous space using a data-driven approach for the 3D object reconstruction task. We have validated the proposed approach making use of two groups of experiments. In the first group, several variants of the proposed architecture are analyzed. Predicted next-best-views were observed to be closely positioned to the ground truth. In the second group of experiments, the proposed approach is requested to reconstruct several unseen objects, namely, objects not considered by the 3D CNN during training nor validation. Coverage percentages of up to 90 \% were observed. With respect to current state-of-the-art methods, the proposed approach improves the performance of previous next-best-view classification approaches and it is quite fast in running time (3 frames per second), given that it does not compute the expensive ray tracing required by previous information metrics.
Channel Estimation for RIS Assisted Wireless Communications: Part II -- An Improved Solution Based on Double-Structured Sparsity
Venue: arXiv cs.IT 2021
Year: 2021

Reconfigurable intelligent surface (RIS) can manipulate the wireless communication environment by controlling the coefficients of RIS elements. However, due to the large number of passive RIS elements without signal processing capability, channel estimation in RIS assisted wireless communication system requires high pilot overhead. In the second part of this invited paper, we propose to exploit the double-structured sparsity of the angular cascaded channels among users to reduce the pilot overhead. Specifically, we first reveal the double-structured sparsity, i.e., different angular cascaded channels for different users enjoy the completely common non-zero rows and the partially common non-zero columns. By exploiting this double-structured sparsity, we further propose the double-structured orthogonal matching pursuit (DS-OMP) algorithm, where the completely common non-zero rows and the partially common non-zero columns are jointly estimated for all users. Simulation results show that the pilot overhead required by the proposed scheme is lower than existing schemes.
Recovery and Analysis of Architecture Descriptions using Centrality Measures
Venue: arXiv cs.SE 2021
Year: 2021

The necessity of an explicit architecture description has been continuously emphasized to communicate the system functionality and for system maintenance activities. This paper presents an approach to extract architecture descriptions using the {\em centrality measures} from the theory of Social Network Analysis. The architecture recovery approach presented in this paper works in two phases. The first phase aims to calculate centrality measures for each program element in the system. The second phase assumes that the system has been designed around the layered architecture style and assigns layers to each program element. Two techniques to assign program elements are presented. The first technique of layer assignment uses a set of pre-defined rules, while the second technique learns the rules of assignment from a pre-labelled data set. The paper presents the evaluation of both approaches.
A Software Architecture Teacher's Dilemmas
Venue: arXiv cs.SE 2021
Year: 2021

An instructor teaching a course on Software Architecture needs to be more reflective to engage students productively in the learning activities. In this reflective essay, the author identifies a few decisive moments referred to as instructional dilemmas at which a teacher reflects upon choices and their consequences so that meaningful learning happens. These situations are referred to as dilemmas because they offer two options to instructors. Some of these dilemmas arise from the inherent nature of Software Architecture as a discipline, while the source of others is the background knowledge of learners. The paper suggests a set of principles and small-teaching methods to make teaching and learning more effective in such situations.
An Optimal Reduction of TV-Denoising to Adaptive Online Learning
Venue: arXiv cs.LG 2021
Year: 2021

We consider the problem of estimating a function from $n$ noisy samples whose discrete Total Variation (TV) is bounded by $C_n$. We reveal a deep connection to the seemingly disparate problem of Strongly Adaptive online learning (Daniely et al, 2015) and provide an $O(n \log n)$ time algorithm that attains the near minimax optimal rate of $\tilde O (n^{1/3}C_n^{2/3})$ under squared error loss. The resulting algorithm runs online and optimally adapts to the unknown smoothness parameter $C_n$. This leads to a new and more versatile alternative to wavelets-based methods for (1) adaptively estimating TV bounded functions; (2) online forecasting of TV bounded trends in time series.
Error Diffusion Halftoning Against Adversarial Examples
Venue: arXiv cs.CV 2021
Year: 2021

Adversarial examples contain carefully crafted perturbations that can fool deep neural networks (DNNs) into making wrong predictions. Enhancing the adversarial robustness of DNNs has gained considerable interest in recent years. Although image transformation-based defenses were widely considered at an earlier time, most of them have been defeated by adaptive attacks. In this paper, we propose a new image transformation defense based on error diffusion halftoning, and combine it with adversarial training to defend against adversarial examples. Error diffusion halftoning projects an image into a 1-bit space and diffuses quantization error to neighboring pixels. This process can remove adversarial perturbations from a given image while maintaining acceptable image quality in the meantime in favor of recognition. Experimental results demonstrate that the proposed method is able to improve adversarial robustness even under advanced adaptive attacks, while most of the other image transformation-based defenses do not. We show that a proper image transformation can still be an effective defense approach. Code: https://github.com/shaoyuanlo/Halftoning-Defense