OpenRaiser/Intern-Atlas huggingface dataset:
Block-SCL: Blocking Matters for Supervised Contrastive Learning in Product Matching
Venue: arXiv cs.CL 2022
Year: 2022

Product matching is a fundamental step for the global understanding of consumer behavior in e-commerce. In practice, product matching refers to the task of deciding if two product offers from different data sources (e.g. retailers) represent the same product. Standard pipelines use a previous stage called blocking, where for a given product offer a set of potential matching candidates are retrieved based on similar characteristics (e.g. same brand, category, flavor, etc.). From these similar product candidates, those that are not a match can be considered hard negatives. We present Block-SCL, a strategy that uses the blocking output to make the most of Supervised Contrastive Learning (SCL). Concretely, Block-SCL builds enriched batches using the hard-negatives samples obtained in the blocking stage. These batches provide a strong training signal leading the model to learn more meaningful sentence embeddings for product matching. Experimental results in several public datasets demonstrate that Block-SCL achieves state-of-the-art results despite only using short product titles as input, no data augmentation, and a lighter transformer backbone than competing methods.
Data Integrity Error Localization in Networked Systems with Missing Data
Venue: arXiv cs.NI 2022
Year: 2022

Most recent network failure diagnosis systems focused on data center networks where complex measurement systems can be deployed to derive routing information and ensure network coverage in order to achieve accurate and fast fault localization. In this paper, we target wide-area networks that support data-intensive distributed applications. We first present a new multi-output prediction model that directly maps the application level observations to localize the system component failures. In reality, this application-centric approach may face the missing data challenge as some input (feature) data to the inference models may be missing due to incomplete or lost measurements in wide area networks. We show that the presented prediction model naturally allows the {\it multivariate} imputation to recover the missing data. We evaluate multiple imputation algorithms and show that the prediction performance can be improved significantly in a large-scale network. As far as we know, this is the first study on the missing data issue and applying imputation techniques in network failure localization.
Community Size and User Migration: Population Model Based on opinion Dynamics
Venue: arXiv cs.SI 2022
Year: 2022

Due to its significance in the recommendation system and community operations, user migration has garnered the interest of cyber-demography experts from numerous disciplines. However, contemporary research frequently overlooks the theory behind related prediction techniques, such as the Hidden Markov model. By combining the two fundamental processes of "opinion evolution" and "individual migration" in this research, the mechanistic explanation of online user migration is established and merged into a composite model. Simultaneously, some fundamental theorems and exploratory conclusions related to our model's consensus and steady population state are established via theoretical proof and numerical simulation.
Tackling Real-World Autonomous Driving using Deep Reinforcement Learning
Venue: arXiv cs.RO 2022
Year: 2022

In the typical autonomous driving stack, planning and control systems represent two of the most crucial components in which data retrieved by sensors and processed by perception algorithms are used to implement a safe and comfortable self-driving behavior. In particular, the planning module predicts the path the autonomous car should follow taking the correct high-level maneuver, while control systems perform a sequence of low-level actions, controlling steering angle, throttle and brake. In this work, we propose a model-free Deep Reinforcement Learning Planner training a neural network that predicts both acceleration and steering angle, thus obtaining a single module able to drive the vehicle using the data processed by localization and perception algorithms on board of the self-driving car. In particular, the system that was fully trained in simulation is able to drive smoothly and safely in obstacle-free environments both in simulation and in a real-world urban area of the city of Parma, proving that the system features good generalization capabilities also driving in those parts outside the training scenarios. Moreover, in order to deploy the system on board of the real self-driving car and to reduce the gap between simulated and real-world performances, we also develop a module represented by a tiny neural network able to reproduce the real vehicle dynamic behavior during the training in simulation.
CLEAR: Improving Vision-Language Navigation with Cross-Lingual, Environment-Agnostic Representations
Venue: arXiv cs.CV 2022
Year: 2022

Vision-and-Language Navigation (VLN) tasks require an agent to navigate through the environment based on language instructions. In this paper, we aim to solve two key challenges in this task: utilizing multilingual instructions for improved instruction-path grounding and navigating through new environments that are unseen during training. To address these challenges, we propose CLEAR: Cross-Lingual and Environment-Agnostic Representations. First, our agent learns a shared and visually-aligned cross-lingual language representation for the three languages (English, Hindi and Telugu) in the Room-Across-Room dataset. Our language representation learning is guided by text pairs that are aligned by visual information. Second, our agent learns an environment-agnostic visual representation by maximizing the similarity between semantically-aligned image pairs (with constraints on object-matching) from different environments. Our environment agnostic visual representation can mitigate the environment bias induced by low-level visual information. Empirically, on the Room-Across-Room dataset, we show that our multilingual agent gets large improvements in all metrics over the strong baseline model when generalizing to unseen environments with the cross-lingual language representation and the environment-agnostic visual representation. Furthermore, we show that our learned language and visual representations can be successfully transferred to the Room-to-Room and Cooperative Vision-and-Dialogue Navigation task, and present detailed qualitative and quantitative generalization and grounding analysis. Our code is available at https://github.com/jialuli-luka/CLEAR
Cooperate or Compete: A New Perspective on Training of Generative Networks
Venue: arXiv cs.LG 2022
Year: 2022

GANs have two competing modules: the generator module is trained to generate new examples, and the discriminator module is trained to discriminate real examples from generated examples. The training procedure of GAN is modeled as a finitely repeated simultaneous game. Each module tries to increase its performance at every repetition of the base game (at every batch of training data) in a non-cooperative manner. We observed that each module can perform better and learn faster if training is modeled as an infinitely repeated simultaneous game. At every repetition of the base game (at every batch of training data) the stronger module (whose performance is increased or remains the same compared to the previous batch of training data) cooperates with the weaker module (whose performance is decreased compared to the previous batch of training data) and only the weaker module is allowed to increase its performance.
Learning Classifier Systems for Self-Explaining Socio-Technical-Systems
Venue: arXiv cs.HC 2022
Year: 2022

In socio-technical settings, operators are increasingly assisted by decision support systems. By employing these, important properties of socio-technical systems such as self-adaptation and self-optimization are expected to improve further. To be accepted by and engage efficiently with operators, decision support systems need to be able to provide explanations regarding the reasoning behind specific decisions. In this paper, we propose the usage of Learning Classifier Systems, a family of rule-based machine learning methods, to facilitate transparent decision making and highlight some techniques to improve that. We then present a template of seven questions to assess application-specific explainability needs and demonstrate their usage in an interview-based case study for a manufacturing scenario. We find that the answers received did yield useful insights for a well-designed LCS model and requirements to have stakeholders actively engage with an intelligent agent.
Generalization to translation shifts: a study in architectures and augmentations
Venue: arXiv cs.CV 2022
Year: 2022

We study how effective data augmentation is at capturing the inductive bias of carefully designed network architectures for spatial translation invariance. We evaluate various image classification architectures (antialiased, convolutional, vision transformer, and fully connected MLP networks) and data augmentation techniques towards generalization to large translation shifts. We observe that: (a) without data augmentation, all architectures, including convolutional networks with antialiased modification suffer some degradation in performance when evaluated on translated test distributions. Understandably, both the in-distribution accuracy and degradation to shifts is significantly worse for non-convolutional models. (b) The robustness of performance is improved by even a minimal augmentation of $4$ pixel random crop across all architectures. In some instances, even $1-2$ pixel random crop is sufficient. This suggests that there is a form of meta generalization from augmentation. For non-convolutional architectures, while the absolute accuracy is still low with this basic augmentation, we see substantial improvements in robustness to translation shifts. (c) With a sufficiently advanced augmentation pipeline ($4$ pixel crop+RandAugmentation+Erasing+MixUp), all architectures can be trained to have competitive performance in terms of in-distribution accuracy as well as generalization to large translation shifts.
Zero-shot Cross-Linguistic Learning of Event Semantics
Venue: arXiv cs.CL 2022
Year: 2022

Typologically diverse languages offer systems of lexical and grammatical aspect that allow speakers to focus on facets of event structure in ways that comport with the specific communicative setting and discourse constraints they face. In this paper, we look specifically at captions of images across Arabic, Chinese, Farsi, German, Russian, and Turkish and describe a computational model for predicting lexical aspects. Despite the heterogeneity of these languages, and the salient invocation of distinctive linguistic resources across their caption corpora, speakers of these languages show surprising similarities in the ways they frame image content. We leverage this observation for zero-shot cross-lingual learning and show that lexical aspects can be predicted for a given language despite not having observed any annotated data for this language at all.
Ensemble feature selection with clustering for analysis of high-dimensional, correlated clinical data in the search for Alzheimer's disease biomarkers
Venue: arXiv cs.LG 2022
Year: 2022

Healthcare datasets often contain groups of highly correlated features, such as features from the same biological system. When feature selection is applied to these datasets to identify the most important features, the biases inherent in some multivariate feature selectors due to correlated features make it difficult for these methods to distinguish between the important and irrelevant features and the results of the feature selection process can be unstable. Feature selection ensembles, which aggregate the results of multiple individual base feature selectors, have been investigated as a means of stabilising feature selection results, but do not address the problem of correlated features. We present a novel framework to create feature selection ensembles from multivariate feature selectors while taking into account the biases produced by groups of correlated features, using agglomerative hierarchical clustering in a pre-processing step. These methods were applied to two real-world datasets from studies of Alzheimer's disease (AD), a progressive neurodegenerative disease that has no cure and is not yet fully understood. Our results show a marked improvement in the stability of features selected over the models without clustering, and the features selected by these models are in keeping with the findings in the AD literature.
BioTABQA: Instruction Learning for Biomedical Table Question Answering
Venue: arXiv cs.CL 2022
Year: 2022

Table Question Answering (TQA) is an important but under-explored task. Most of the existing QA datasets are in unstructured text format and only few of them use tables as the context. To the best of our knowledge, none of TQA datasets exist in the biomedical domain where tables are frequently used to present information. In this paper, we first curate a table question answering dataset, BioTABQA, using 22 templates and the context from a biomedical textbook on differential diagnosis. BioTABQA can not only be used to teach a model how to answer questions from tables but also evaluate how a model generalizes to unseen questions, an important scenario for biomedical applications. To achieve the generalization evaluation, we divide the templates into 17 training and 5 cross-task evaluations. Then, we develop two baselines using single and multi-tasks learning on BioTABQA. Furthermore, we explore instructional learning, a recent technique showing impressive generalizing performance. Experimental results show that our instruction-tuned model outperforms single and multi-task baselines on an average by ~23% and ~6% across various evaluation settings, and more importantly, instruction-tuned model outperforms baselines by ~5% on cross-tasks.
DCT-Net: Domain-Calibrated Translation for Portrait Stylization
Venue: arXiv cs.CV 2022
Year: 2022

This paper introduces DCT-Net, a novel image translation architecture for few-shot portrait stylization. Given limited style exemplars ($\sim$100), the new architecture can produce high-quality style transfer results with advanced ability to synthesize high-fidelity contents and strong generality to handle complicated scenes (e.g., occlusions and accessories). Moreover, it enables full-body image translation via one elegant evaluation network trained by partial observations (i.e., stylized heads). Few-shot learning based style transfer is challenging since the learned model can easily become overfitted in the target domain, due to the biased distribution formed by only a few training examples. This paper aims to handle the challenge by adopting the key idea of "calibration first, translation later" and exploring the augmented global structure with locally-focused translation. Specifically, the proposed DCT-Net consists of three modules: a content adapter borrowing the powerful prior from source photos to calibrate the content distribution of target samples; a geometry expansion module using affine transformations to release spatially semantic constraints; and a texture translation module leveraging samples produced by the calibrated distribution to learn a fine-grained conversion. Experimental results demonstrate the proposed method's superiority over the state of the art in head stylization and its effectiveness on full image translation with adaptive deformations.
You have been warned: Abusing 5G's Warning and Emergency Systems
Venue: arXiv cs.CR 2022
Year: 2022

The Public Warning System (PWS) is an essential part of cellular networks and a country's civil protection. Warnings can notify users of hazardous events (e.g., floods, earthquakes) and crucial national matters that require immediate attention. PWS attacks disseminating fake warnings or concealing precarious events can have a serious impact, causing fraud, panic, physical harm, or unrest to users within an affected area. In this work, we conduct the first comprehensive investigation of PWS security in 5G networks. We demonstrate five practical attacks that may impact the security of 5G-based Commercial Mobile Alert System (CMAS) as well as Earthquake and Tsunami Warning System (ETWS) alerts. Additional to identifying the vulnerabilities, we investigate two PWS spoofing and three PWS suppression attacks, with or without a man-in-the-middle (MitM) attacker. We discover that MitM-based attacks have more severe impact than their non-MitM counterparts. Our PWS barring attack is an effective technique to eliminate legitimate warning messages. We perform a rigorous analysis of the roaming aspect of the PWS, incl. its potentially secure version, and report the implications of our attacks on other emergency features (e.g., 911 SIP calls). We discuss possible countermeasures and note that eradicating the attacks necessitates a scrupulous reevaluation of the PWS design and a secure implementation.
Learning to Diversify for Product Question Generation
Venue: arXiv cs.CL 2022
Year: 2022

We address the product question generation task. For a given product description, our goal is to generate questions that reflect potential user information needs that are either missing or not well covered in the description. Moreover, we wish to cover diverse user information needs that may span a multitude of product types. To this end, we first show how the T5 pre-trained Transformer encoder-decoder model can be fine-tuned for the task. Yet, while the T5 generated questions have a reasonable quality compared to the state-of-the-art method for the task (KPCNet), many of such questions are still too general, resulting in a sub-optimal global question diversity. As an alternative, we propose a novel learning-to-diversify (LTD) fine-tuning approach that allows to enrich the language learned by the underlying Transformer model. Our empirical evaluation shows that, using our approach significantly improves the global diversity of the underlying Transformer model, while preserves, as much as possible, its generation relevance.
Transformers discover an elementary calculation system exploiting local attention and grid-like problem representation
Venue: arXiv cs.LG 2022
Year: 2022

Mathematical reasoning is one of the most impressive achievements of human intellect but remains a formidable challenge for artificial intelligence systems. In this work we explore whether modern deep learning architectures can learn to solve a symbolic addition task by discovering effective arithmetic procedures. Although the problem might seem trivial at first glance, generalizing arithmetic knowledge to operations involving a higher number of terms, possibly composed by longer sequences of digits, has proven extremely challenging for neural networks. Here we show that universal transformers equipped with local attention and adaptive halting mechanisms can learn to exploit an external, grid-like memory to carry out multi-digit addition. The proposed model achieves remarkable accuracy even when tested with problems requiring extrapolation outside the training distribution; most notably, it does so by discovering human-like calculation strategies such as place value alignment.
A Direct Construction of Cross Z-Complementary Sequence Sets with Large Set Size
Venue: arXiv cs.IT 2022
Year: 2022

This letter presents a direct construction of cross Z-complementary sequence sets (CZCSSs), whose aperiodic correlation sums exhibit zero correlation zones at both the front-end and tail-end shifts. CZCSS can be regarded as an extension of the symmetrical Z-complementary code set (SZCCS). The available construction of SZCCS has a limitation on the set size, with a maximum set size of 8. The proposed generalized Boolean function based construction can generate CZCSS of length in the form of non-power-of-two with variable set size $2^{n+1}$, where each code has $2^{n+1}$ constituent sequences. The proposed construction also yields cross Z-complementary pairs and cross Z-complementary sets with a large number of constituent sequences compared to the existing work.
Cascaded Deep Hybrid Models for Multistep Household Energy Consumption Forecasting
Venue: arXiv cs.LG 2022
Year: 2022

Sustainability requires increased energy efficiency with minimal waste. The future power systems should thus provide high levels of flexibility iin controling energy consumption. Precise projections of future energy demand/load at the aggregate and on the individual site levels are of great importance for decision makers and professionals in the energy industry. Forecasting energy loads has become more advantageous for energy providers and customers, allowing them to establish an efficient production strategy to satisfy demand. This study introduces two hybrid cascaded models for forecasting multistep household power consumption in different resolutions. The first model integrates Stationary Wavelet Transform (SWT), as an efficient signal preprocessing technique, with Convolutional Neural Networks and Long Short Term Memory (LSTM). The second hybrid model combines SWT with a self-attention based neural network architecture named transformer. The major constraint of using time-frequency analysis methods such as SWT in multistep energy forecasting problems is that they require sequential signals, making signal reconstruction problematic in multistep forecasting applications.The cascaded models can efficiently address this problem through using the recursive outputs. Experimental results show that the proposed hybrid models achieve superior prediction performance compared to the existing multistep power consumption prediction methods. The results will pave the way for more accurate and reliable forecasting of household power consumption.
Massive RF Simulation Applied to School Connectivity in Malawi
Venue: arXiv cs.NI 2022
Year: 2022

Providing Internet connectivity to schools has been identified as paramount for development, for instance in the Giga project, cosponsored by ITU and UNICEF, with the goal of connecting every school to the Internet by 2030. For a country wide deployment, it is imperative to perform a thorough planning of the whole installation, using radio frequency (RF) propagation models. While statistical models based on empirical RF propagation data gathered in different scenarios can be employed, for point to point links at microwave frequencies the existence of a clear line of sight (LOS) is normally a prerequisite. The Irregular terrain model which makes use of digital elevation maps (DEM) has proved quite effective for simulating point to point links, but its application to a great number of links becomes time consuming, so we have developed an automated framework to perform this task. As a case study we have applied this framework in the planning of a project mired at providing connectivity to primary and secondary schools all over the country of Malawi.