OpenRaiser/Intern-Atlas huggingface dataset:
Language model acceptability judgements are not always robust to context
Venue: arXiv cs.CL 2022
Year: 2022

Targeted syntactic evaluations of language models ask whether models show stable preferences for syntactically acceptable content over minimal-pair unacceptable inputs. Most targeted syntactic evaluation datasets ask models to make these judgements with just a single context-free sentence as input. This does not match language models' training regime, in which input sentences are always highly contextualized by the surrounding corpus. This mismatch raises an important question: how robust are models' syntactic judgements in different contexts? In this paper, we investigate the stability of language models' performance on targeted syntactic evaluations as we vary properties of the input context: the length of the context, the types of syntactic phenomena it contains, and whether or not there are violations of grammaticality. We find that model judgements are generally robust when placed in randomly sampled linguistic contexts. However, they are substantially unstable for contexts containing syntactic structures matching those in the critical test content. Among all tested models (GPT-2 and five variants of OPT), we significantly improve models' judgements by providing contexts with matching syntactic structures, and conversely significantly worsen them using unacceptable contexts with matching but violated syntactic structures. This effect is amplified by the length of the context, except for unrelated inputs. We show that these changes in model performance are not explainable by simple features matching the context and the test inputs, such as lexical overlap and dependency overlap. This sensitivity to highly specific syntactic features of the context can only be explained by the models' implicit in-context learning abilities.
Impact of Sentiment Analysis in Fake Review Detection
Venue: arXiv cs.CL 2022
Year: 2022

Fake review identification is an important topic and has gained the interest of experts all around the world. Identifying fake reviews is challenging for researchers, and there are several primary challenges to fake review detection. We propose developing an initial research paper for investigating fake reviews by using sentiment analysis. Ten research papers are identified that show fake reviews, and they discuss currently available solutions for predicting or detecting fake reviews. They also show the distribution of fake and truthful reviews through the analysis of sentiment. We summarize and compare previous studies related to fake reviews. We highlight the most significant challenges in the sentiment evaluation process and demonstrate that there is a significant impact on sentiment scores used to identify fake feedback.
Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search
Venue: arXiv cs.IR 2022
Year: 2022

Medical systematic reviews typically require assessing all the documents retrieved by a search. The reason is two-fold: the task aims for ``total recall''; and documents retrieved using Boolean search are an unordered set, and thus it is unclear how an assessor could examine only a subset. Screening prioritisation is the process of ranking the (unordered) set of retrieved documents, allowing assessors to begin the downstream processes of the systematic review creation earlier, leading to earlier completion of the review, or even avoiding screening documents ranked least relevant. Screening prioritisation requires highly effective ranking methods. Pre-trained language models are state-of-the-art on many IR tasks but have yet to be applied to systematic review screening prioritisation. In this paper, we apply several pre-trained language models to the systematic review document ranking task, both directly and fine-tuned. An empirical analysis compares how effective neural methods compare to traditional methods for this task. We also investigate different types of document representations for neural methods and their impact on ranking performance. Our results show that BERT-based rankers outperform the current state-of-the-art screening prioritisation methods. However, BERT rankers and existing methods can actually be complementary, and thus, further improvements may be achieved if used in conjunction.
AutoSlicer: Scalable Automated Data Slicing for ML Model Analysis
Venue: arXiv cs.LG 2022
Year: 2022

Automated slicing aims to identify subsets of evaluation data where a trained model performs anomalously. This is an important problem for machine learning pipelines in production since it plays a key role in model debugging and comparison, as well as the diagnosis of fairness issues. Scalability has become a critical requirement for any automated slicing system due to the large search space of possible slices and the growing scale of data. We present Autoslicer, a scalable system that searches for problematic slices through distributed metric computation and hypothesis testing. We develop an efficient strategy that reduces the search space through pruning and prioritization. In the experiments, we show that our search strategy finds most of the anomalous slices by inspecting a small portion of the search space.
Minimizing Maximum Model Discrepancy for Transferable Black-box Targeted Attacks
Venue: arXiv cs.CV 2022
Year: 2022

In this work, we study the black-box targeted attack problem from the model discrepancy perspective. On the theoretical side, we present a generalization error bound for black-box targeted attacks, which gives a rigorous theoretical analysis for guaranteeing the success of the attack. We reveal that the attack error on a target model mainly depends on empirical attack error on the substitute model and the maximum model discrepancy among substitute models. On the algorithmic side, we derive a new algorithm for black-box targeted attacks based on our theoretical analysis, in which we additionally minimize the maximum model discrepancy(M3D) of the substitute models when training the generator to generate adversarial examples. In this way, our model is capable of crafting highly transferable adversarial examples that are robust to the model variation, thus improving the success rate for attacking the black-box model. We conduct extensive experiments on the ImageNet dataset with different classification models, and our proposed approach outperforms existing state-of-the-art methods by a significant margin. Our codes will be released.
Automated Optical Inspection of FAST's Reflector Surface using Drones and Computer Vision
Venue: arXiv cs.CV 2022
Year: 2022

The Five-hundred-meter Aperture Spherical radio Telescope (FAST) is the world's largest single-dish radio telescope. Its large reflecting surface achieves unprecedented sensitivity but is prone to damage, such as dents and holes, caused by naturally-occurring falling objects. Hence, the timely and accurate detection of surface defects is crucial for FAST's stable operation. Conventional manual inspection involves human inspectors climbing up and examining the large surface visually, a time-consuming and potentially unreliable process. To accelerate the inspection process and increase its accuracy, this work makes the first step towards automating the inspection of FAST by integrating deep-learning techniques with drone technology. First, a drone flies over the surface along a predetermined route. Since surface defects significantly vary in scale and show high inter-class similarity, directly applying existing deep detectors to detect defects on the drone imagery is highly prone to missing and misidentifying defects. As a remedy, we introduce cross-fusion, a dedicated plug-in operation for deep detectors that enables the adaptive fusion of multi-level features in a point-wise selective fashion, depending on local defect patterns. Consequently, strong semantics and fine-grained details are dynamically fused at different positions to support the accurate detection of defects of various scales and types. Our AI-powered drone-based automated inspection is time-efficient, reliable, and has good accessibility, which guarantees the long-term and stable operation of FAST.
Gait Recognition Using 3-D Human Body Shape Inference
Venue: arXiv cs.CV 2022
Year: 2022

Gait recognition, which identifies individuals based on their walking patterns, is an important biometric technique since it can be observed from a distance and does not require the subject's cooperation. Recognizing a person's gait is difficult because of the appearance variants in human silhouette sequences produced by varying viewing angles, carrying objects, and clothing. Recent research has produced a number of ways for coping with these variants. In this paper, we present the usage of inferring 3-D body shapes distilled from limited images, which are, in principle, invariant to the specified variants. Inference of 3-D shape is a difficult task, especially when only silhouettes are provided in a dataset. We provide a method for learning 3-D body inference from silhouettes by transferring knowledge from 3-D shape prior from RGB photos. We use our method on multiple existing state-of-the-art gait baselines and obtain consistent improvements for gait identification on two public datasets, CASIA-B and OUMVLP, on several variants and settings, including a new setting of novel views not seen during training.
A Degree Based Approximation of an SIR Model with Contact Tracing and Isolation
Venue: arXiv cs.SI 2022
Year: 2022

In this paper we study a susceptible infectious recovered (SIR) model with asymptomatic patients, contact tracing and isolation on a configuration network. Using degree based approximation, we derive a system of differential equations for this model. This system can not be solved analytically. We present an early-time analysis for the model. The early-time analysis produces an epidemic threshold. On one side of the threshold, the disease dies out quickly. On the other side, a significant fraction of the population are infected. The threshold only depends on the parameters of the disease, the mean access degree of the network, and the fraction of asymptomatic patients. The threshold does not depend on the parameter of contact tracing and isolation policy. We present an approximate analysis which greatly reduces computational complexity. The nonlinear system derived from the approximation is not almost linear. We present a stability analysis for this system. We simulate the SIR model with contact tracing and isolation on five real-world networks. Simulation results show that contact tracing and isolation are useful to contain epidemics.
Efficient RDMA Communication Protocols
Venue: arXiv cs.NI 2022
Year: 2022

Developers of networked systems often work with low-level RDMA libraries to tailor network modules to take full advantage of offload capabilities offered by RDMA-capable network controllers. Because of the huge design space of networked data access protocols and variability in capabilities of RDMA infrastructure, developers tend to reinvent and reimplement common data exchange protocols, wasting months of development yet missing various performance and system capabilities. In this work, we summarise and categorize RDMA data exchange protocols and elaborate on what features they can offer to networked systems and what implications they have on their memory and network management.
Natural Language to Code Generation in Interactive Data Science Notebooks
Venue: arXiv cs.CL 2022
Year: 2022

Computational notebooks, such as Jupyter notebooks, are interactive computing environments that are ubiquitous among data scientists to perform data wrangling and analytic tasks. To measure the performance of AI pair programmers that automatically synthesize programs for those tasks given natural language (NL) intents from users, we build ARCADE, a benchmark of 1082 code generation problems using the pandas data analysis framework in data science notebooks. ARCADE features multiple rounds of NL-to-code problems from the same notebook. It requires a model to understand rich multi-modal contexts, such as existing notebook cells and their execution states as well as previous turns of interaction. To establish a strong baseline on this challenging task, we develop PaChiNCo, a 62B code language model (LM) for Python computational notebooks, which significantly outperforms public code LMs. Finally, we explore few-shot prompting strategies to elicit better code with step-by-step decomposition and NL explanation, showing the potential to improve the diversity and explainability of model predictions.
Mind the Knowledge Gap: A Survey of Knowledge-enhanced Dialogue Systems
Venue: arXiv cs.CL 2022
Year: 2022

Many dialogue systems (DSs) lack characteristics humans have, such as emotion perception, factuality, and informativeness. Enhancing DSs with knowledge alleviates this problem, but, as many ways of doing so exist, keeping track of all proposed methods is difficult. Here, we present the first survey of knowledge-enhanced DSs. We define three categories of systems - internal, external, and hybrid - based on the knowledge they use. We survey the motivation for enhancing DSs with knowledge, used datasets, and methods for knowledge search, knowledge encoding, and knowledge incorporation. Finally, we propose how to improve existing systems based on theories from linguistics and cognitive science.
Diversity Analysis of Multi-Aperture UWOC System over EGG Channel with Pointing Errors
Venue: arXiv cs.IT 2022
Year: 2022

Single aperture reception for underwater wireless optical communication (UWOC) is insufficient to deal with oceanic turbulence caused by the combined effect of temperature gradient and air bubbles. This paper analyzes the performance of multi-aperture reception for UWOC under channel irradiance fluctuations characterized by the mixture exponential generalized gamma (EGG) distribution. We analyze the system performance by employing both selection combining (SC) and maximum ratio combining (MRC) receivers. In particular, we derive the exact outage probability expression for the SC-based multi-aperture UWOC receiver and obtain an upper bound on the outage probability for the MRC-based multi-aperture UWOC receiver. With the help of the derived results, we analytically obtain the diversity order of the considered multi-aperture UWOC system.
MIGA: A Unified Multi-task Generation Framework for Conversational Text-to-SQL
Venue: arXiv cs.CL 2022
Year: 2022

Conversational text-to-SQL is designed to translate multi-turn natural language questions into their corresponding SQL queries. Most state-of-the-art conversational text- to-SQL methods are incompatible with generative pre-trained language models (PLMs), such as T5. In this paper, we present a two-stage unified MultI-task Generation frAmework (MIGA) that leverages PLMs' ability to tackle conversational text-to-SQL. In the pre-training stage, MIGA first decomposes the main task into several related sub-tasks and then unifies them into the same sequence-to-sequence (Seq2Seq) paradigm with task-specific natural language prompts to boost the main task from multi-task training. Later in the fine-tuning stage, we propose four SQL perturbations to alleviate the error propagation problem. MIGA tends to achieve state-of-the-art performance on two benchmarks (SparC and CoSQL). We also provide extensive analyses and discussions to shed light on some new perspectives for conversational text-to-SQL.
An Investigation of Indian Native Language Phonemic Influences on L2 English Pronunciations
Venue: arXiv cs.CL 2022
Year: 2022

Speech systems are sensitive to accent variations. This is especially challenging in the Indian context, with an abundance of languages but a dearth of linguistic studies characterising pronunciation variations. The growing number of L2 English speakers in India reinforces the need to study accents and L1-L2 interactions. We investigate the accents of Indian English (IE) speakers and report in detail our observations, both specific and common to all regions. In particular, we observe the phonemic variations and phonotactics occurring in the speakers' native languages and apply this to their English pronunciations. We demonstrate the influence of 18 Indian languages on IE by comparing the native language pronunciations with IE pronunciations obtained jointly from existing literature studies and phonetically annotated speech of 80 speakers. Consequently, we are able to validate the intuitions of Indian language influences on IE pronunciations by justifying pronunciation rules from the perspective of Indian language phonology. We obtain a comprehensive description in terms of universal and region-specific characteristics of IE, which facilitates accent conversion and adaptation of existing ASR and TTS systems to different Indian accents.
Enabling Temporal-Spectral Decoding in Pre-movement Detection
Venue: arXiv cs.HC 2022
Year: 2022

Non-invasive brain-computer interfaces help the subjects to control external devices by brain intentions. The multi-class classification of upper limb movements can provide external devices with more control commands. The onsets of the upper limb movements are located by the external limb trajectory to eliminate the delay and bias among the trials. However, the trajectories are not recorded due to the limitation of experiments. The delay cannot be avoided in the analysis of signals. The delay negatively influences the classification performance, which limits the further application of upper limb movements in the brain-computer interface. This work focuses on multi-channel brain signals analysis in the temporal-frequency approach. It proposes the two-stage-training temporal-spectral neural network (TTSNet) to decode patterns from brain signals. The TTSNet first divides the signals into various filter banks. In each filter bank, task-related component analysis is used to reduce the dimension and reject the noise of the brain. A convolutional neural network (CNN) is then used to optimize the temporal characteristic of signals and extract class-related features. Finally, these class-related features from all filter banks are fused by concatenation and classified by the fully connected layer of the CNN. The proposed method is evaluated in two public datasets. The results show that TTSNet has an improved accuracy of 0.7456$\pm$0.1205 compared to the EEGNet of 0.6506$\pm$0.1275 ($p<0.05$) and FBTRCA of 0.6787$\pm$0.1260 ($p<0.1$) in the movement detection task, which classifies the movement state and the resting state. The proposed method is expected to help detect limb movements and assist in the rehabilitation of stroke patients.
Real-Time Rendering of Arbitrary Surface Geometries using Learnt Transfer
Venue: arXiv cs.GR 2022
Year: 2022

Precomputed Radiance Transfer (PRT) is widely used for real-time photorealistic effects. PRT disentangles the rendering equation into transfer and lighting, enabling their precomputation. Transfer accounts for the cosine-weighted visibility of points in the scene while lighting for emitted radiance from the environment. Prior art stored precomputed transfer in a tabulated manner, either in vertex or texture space. These values are fetched with interpolation at each point for shading. Vertex space methods require densely tessellated mesh vertices for high quality images. Texture space methods require non-overlapping and area-preserving UV mapping to be available. They also require a high-resolution texture to avoid rendering artifacts. In this paper, we propose a compact transfer representation that is learnt directly on scene geometry points. Specifically, we train a small multi-layer perceptron (MLP) to predict the transfer at sampled surface points. Our approach is most beneficial where inherent mesh storage structure and natural UV mapping are not available, such as Implicit Surfaces as it learns the transfer values directly on the surface. We demonstrate real-time, photorealistic renderings of diffuse and glossy materials on SDF geometries with PRT using our approach.
Synthetic Data Augmentation Using GAN For Improved Automated Visual Inspection
Venue: arXiv cs.CV 2022
Year: 2022

Quality control is a crucial activity performed by manufacturing companies to ensure their products conform to the requirements and specifications. The introduction of artificial intelligence models enables to automate the visual quality inspection, speeding up the inspection process and ensuring all products are evaluated under the same criteria. In this research, we compare supervised and unsupervised defect detection techniques and explore data augmentation techniques to mitigate the data imbalance in the context of automated visual inspection. Furthermore, we use Generative Adversarial Networks for data augmentation to enhance the classifiers' discriminative performance. Our results show that state-of-the-art unsupervised defect detection does not match the performance of supervised models but can be used to reduce the labeling workload by more than 50%. Furthermore, the best classification performance was achieved considering GAN-based data generation with AUC ROC scores equal to or higher than 0,9898, even when increasing the dataset imbalance by leaving only 25\% of the images denoting defective products. We performed the research with real-world data provided by Philips Consumer Lifestyle BV.
SrTR: Self-reasoning Transformer with Visual-linguistic Knowledge for Scene Graph Generation
Venue: arXiv cs.CV 2022
Year: 2022

Objects in a scene are not always related. The execution efficiency of the one-stage scene graph generation approaches are quite high, which infer the effective relation between entity pairs using sparse proposal sets and a few queries. However, they only focus on the relation between subject and object in triplet set subject entity, predicate entity, object entity, ignoring the relation between subject and predicate or predicate and object, and the model lacks self-reasoning ability. In addition, linguistic modality has been neglected in the one-stage method. It is necessary to mine linguistic modality knowledge to improve model reasoning ability. To address the above-mentioned shortcomings, a Self-reasoning Transformer with Visual-linguistic Knowledge (SrTR) is proposed to add flexible self-reasoning ability to the model. An encoder-decoder architecture is adopted in SrTR, and a self-reasoning decoder is developed to complete three inferences of the triplet set, s+o-p, s+p-o and p+o-s. Inspired by the large-scale pre-training image-text foundation models, visual-linguistic prior knowledge is introduced and a visual-linguistic alignment strategy is designed to project visual representations into semantic spaces with prior knowledge to aid relational reasoning. Experiments on the Visual Genome dataset demonstrate the superiority and fast inference ability of the proposed method.
Lessons from Robot-Assisted Disaster Response Deployments by the German Rescue Robotics Center Task Force
Venue: arXiv cs.RO 2022
Year: 2022

Earthquakes, fire, and floods often cause structural collapses of buildings. The inspection of damaged buildings poses a high risk for emergency forces or is even impossible, though. We present three recent selected missions of the Robotics Task Force of the German Rescue Robotics Center, where both ground and aerial robots were used to explore destroyed buildings. We describe and reflect the missions as well as the lessons learned that have resulted from them. In order to make robots from research laboratories fit for real operations, realistic test environments were set up for outdoor and indoor use and tested in regular exercises by researchers and emergency forces. Based on this experience, the robots and their control software were significantly improved. Furthermore, top teams of researchers and first responders were formed, each with realistic assessments of the operational and practical suitability of robotic systems.