OpenRaiser/Intern-Atlas huggingface dataset:
Approaching the Soundness Barrier: A Near Optimal Analysis of the Cube versus Cube Test
Venue: arXiv cs.CC 2022
Year: 2022

The Cube versus Cube test is a variant of the well-known Plane versus Plane test of Raz and Safra, in which to each $3$-dimensional affine subspace $C$ of $\mathbb{F}_q^n$, a polynomial of degree at most $d$, $T(C)$, is assigned in a somewhat locally consistent manner: taking two cubes $C_1, C_2$ that intersect in a plane uniformly at random, the probability that $T(C_1)$ and $T(C_2)$ agree on $C_1\cap C_2$ is at least some $ε$. An element of interest is the soundness threshold of this test, i.e. the smallest value of $ε$, such that this amount of local consistency implies a global structure; namely, that there is a global degree $d$ function $g$ such that $g|_{C} \equiv T(C)$ for at least $Ω(ε)$ fraction of the cubes. We show that the cube versus cube low degree test has soundness ${\sf poly}(d)/q$. This result achieves the optimal dependence on $q$ for soundness in low degree testing and improves upon previous soundness results of ${\sf poly}(d)/q^{1/2}$ due to Bhangale, Dinur and Navon.
More Effective Centrality-Based Attacks on Weighted Networks
Venue: arXiv cs.NI 2022
Year: 2022

Only when understanding hackers' tactics, can we thwart their attacks. With this spirit, this paper studies how hackers can effectively launch the so-called 'targeted node attacks', in which iterative attacks are staged on a network, and in each iteration the most important node is removed. In the existing attacks for weighted networks, the node importance is typically measured by the centralities related to shortest-path lengths, and the attack effectiveness is also measured mostly by length-related metrics. However, this paper argues that flows can better reflect network functioning than shortest-path lengths for those networks with carrying traffic as the main functionality. Thus, this paper proposes metrics based on flows for measuring the node importance and the attack effectiveness, respectively. Our node importance metrics include three flow-based centralities (flow betweenness, current-flow betweenness and current-flow closeness), which have not been proposed for use in the attacks on weighted networks yet. Our attack effectiveness metric is a new one proposed by us based on average network flow. Extensive experiments on both artificial and real-world networks show that the attack methods with our three suggested centralities are more effective than the existing attack methods when evaluated under our proposed attack effectiveness metric.
Execution-based Evaluation for Data Science Code Generation Models
Venue: arXiv cs.SE 2022
Year: 2022

Code generation models can benefit data scientists' productivity by automatically generating code from context and text descriptions. An important measure of the modeling progress is whether a model can generate code that can correctly execute to solve the task. However, due to the lack of an evaluation dataset that directly supports execution-based model evaluation, existing work relies on code surface form similarity metrics (e.g., BLEU, CodeBLEU) for model selection, which can be inaccurate. To remedy this, we introduce ExeDS, an evaluation dataset for execution evaluation for data science code generation tasks. ExeDS contains a set of 534 problems from Jupyter Notebooks, each consisting of code context, task description, reference program, and the desired execution output. With ExeDS, we evaluate the execution performance of five state-of-the-art code generation models that have achieved high surface-form evaluation scores. Our experiments show that models with high surface-form scores do not necessarily perform well on execution metrics, and execution-based metrics can better capture model code generation errors. Source code and data can be found at https://github.com/Jun-jie-Huang/ExeDS
Balanced Deep CCA for Bird Vocalization Detection
Venue: arXiv cs.SD 2022
Year: 2022

Event detection improves when events are captured by two different modalities rather than just one. But to train detection systems on multiple modalities is challenging, in particular when there is abundance of unlabelled data but limited amounts of labeled data. We develop a novel self-supervised learning technique for multi-modal data that learns (hidden) correlations between simultaneously recorded microphone (sound) signals and accelerometer (body vibration) signals. The key objective of this work is to learn useful embeddings associated with high performance in downstream event detection tasks when labeled data is scarce and the audio events of interest (songbird vocalizations) are sparse. We base our approach on deep canonical correlation analysis (DCCA) that suffers from event sparseness. We overcome the sparseness of positive labels by first learning a data sampling model from the labelled data and by applying DCCA on the output it produces. This method that we term balanced DCCA (b-DCCA) improves the performance of the unsupervised embeddings on the downstream supervised audio detection task compared to classsical DCCA. Because data labels are frequently imbalanced, our method might be of broad utility in low-resource scenarios.
Data-Efficient Autoregressive Document Retrieval for Fact Verification
Venue: arXiv cs.CL 2022
Year: 2022

Document retrieval is a core component of many knowledge-intensive natural language processing task formulations such as fact verification and question answering. Sources of textual knowledge, such as Wikipedia articles, condition the generation of answers from the models. Recent advances in retrieval use sequence-to-sequence models to incrementally predict the title of the appropriate Wikipedia page given a query. However, this method requires supervision in the form of human annotation to label which Wikipedia pages contain appropriate context. This paper introduces a distant-supervision method that does not require any annotation to train autoregressive retrievers that attain competitive R-Precision and Recall in a zero-shot setting. Furthermore we show that with task-specific supervised fine-tuning, autoregressive retrieval performance for two Wikipedia-based fact verification tasks can approach or even exceed full supervision using less than $1/4$ of the annotated data indicating possible directions for data-efficient autoregressive retrieval.
Physics-Informed Koopman Network
Venue: arXiv cs.LG 2022
Year: 2022

Koopman operator theory is receiving increased attention due to its promise to linearize nonlinear dynamics. Neural networks that are developed to represent Koopman operators have shown great success thanks to their ability to approximate arbitrarily complex functions. However, despite their great potential, they typically require large training data-sets either from measurements of a real system or from high-fidelity simulations. In this work, we propose a novel architecture inspired by physics-informed neural networks, which leverage automatic differentiation to impose the underlying physical laws via soft penalty constraints during model training. We demonstrate that it not only reduces the need of large training data-sets, but also maintains high effectiveness in approximating Koopman eigenfunctions.
FedSiam-DA: Dual-aggregated Federated Learning via Siamese Network under Non-IID Data
Venue: arXiv cs.LG 2022
Year: 2022

Federated learning is a distributed learning that allows each client to keep the original data locally and only upload the parameters of the local model to the server. Despite federated learning can address data island, it remains challenging to train with data heterogeneous in a real application. In this paper, we propose FedSiam-DA, a novel dual-aggregated contrastive federated learning approach, to personalize both local and global models, under various settings of data heterogeneity. Firstly, based on the idea of contrastive learning in the siamese network, FedSiam-DA regards the local and global model as different branches of the siamese network during the local training and controls the update direction of the model by constantly changing model similarity to personalize the local model. Secondly, FedSiam-DA introduces dynamic weights based on model similarity for each local model and exercises the dual-aggregated mechanism to further improve the generalization of the global model. Moreover, we provide extensive experiments on benchmark datasets, the results demonstrate that FedSiam-DA achieves outperforming several previous FL approaches on heterogeneous datasets.
Feedback is Needed for Retakes: An Explainable Poor Image Notification Framework for the Visually Impaired
Venue: arXiv cs.CV 2022
Year: 2022

We propose a simple yet effective image captioning framework that can determine the quality of an image and notify the user of the reasons for any flaws in the image. Our framework first determines the quality of images and then generates captions using only those images that are determined to be of high quality. The user is notified by the flaws feature to retake if image quality is low, and this cycle is repeated until the input image is deemed to be of high quality. As a component of the framework, we trained and evaluated a low-quality image detection model that simultaneously learns difficulty in recognizing images and individual flaws, and we demonstrated that our proposal can explain the reasons for flaws with a sufficient score. We also evaluated a dataset with low-quality images removed by our framework and found improved values for all four common metrics (e.g., BLEU-4, METEOR, ROUGE-L, CIDEr), confirming an improvement in general-purpose image captioning capability. Our framework would assist the visually impaired, who have difficulty judging image quality.
DeepPrivacy2: Towards Realistic Full-Body Anonymization
Venue: arXiv cs.CV 2022
Year: 2022

Generative Adversarial Networks (GANs) are widely adapted for anonymization of human figures. However, current state-of-the-art limit anonymization to the task of face anonymization. In this paper, we propose a novel anonymization framework (DeepPrivacy2) for realistic anonymization of human figures and faces. We introduce a new large and diverse dataset for human figure synthesis, which significantly improves image quality and diversity of generated images. Furthermore, we propose a style-based GAN that produces high quality, diverse and editable anonymizations. We demonstrate that our full-body anonymization framework provides stronger privacy guarantees than previously proposed methods.
Consultation Checklists: Standardising the Human Evaluation of Medical Note Generation
Venue: arXiv cs.CL 2022
Year: 2022

Evaluating automatically generated text is generally hard due to the inherently subjective nature of many aspects of the output quality. This difficulty is compounded in automatic consultation note generation by differing opinions between medical experts both about which patient statements should be included in generated notes and about their respective importance in arriving at a diagnosis. Previous real-world evaluations of note-generation systems saw substantial disagreement between expert evaluators. In this paper we propose a protocol that aims to increase objectivity by grounding evaluations in Consultation Checklists, which are created in a preliminary step and then used as a common point of reference during quality assessment. We observed good levels of inter-annotator agreement in a first evaluation study using the protocol; further, using Consultation Checklists produced in the study as reference for automatic metrics such as ROUGE or BERTScore improves their correlation with human judgements compared to using the original human note.
Progressive Tree-Structured Prototype Network for End-to-End Image Captioning
Venue: arXiv cs.CV 2022
Year: 2022

Studies of image captioning are shifting towards a trend of a fully end-to-end paradigm by leveraging powerful visual pre-trained models and transformer-based generation architecture for more flexible model training and faster inference speed. State-of-the-art approaches simply extract isolated concepts or attributes to assist description generation. However, such approaches do not consider the hierarchical semantic structure in the textual domain, which leads to an unpredictable mapping between visual representations and concept words. To this end, we propose a novel Progressive Tree-Structured prototype Network (dubbed PTSN), which is the first attempt to narrow down the scope of prediction words with appropriate semantics by modeling the hierarchical textual semantics. Specifically, we design a novel embedding method called tree-structured prototype, producing a set of hierarchical representative embeddings which capture the hierarchical semantic structure in textual space. To utilize such tree-structured prototypes into visual cognition, we also propose a progressive aggregation module to exploit semantic relationships within the image and prototypes. By applying our PTSN to the end-to-end captioning framework, extensive experiments conducted on MSCOCO dataset show that our method achieves a new state-of-the-art performance with 144.2% (single model) and 146.5% (ensemble of 4 models) CIDEr scores on `Karpathy' split and 141.4% (c5) and 143.9% (c40) CIDEr scores on the official online test server. Trained models and source code have been released at: https://github.com/NovaMind-Z/PTSN.
An Audit Framework for Technical Assessment of Binary Classifiers
Venue: arXiv cs.CY 2022
Year: 2022

Multilevel models using logistic regression (MLogRM) and random forest models (RFM) are increasingly deployed in industry for the purpose of binary classification. The European Commission's proposed Artificial Intelligence Act (AIA) necessitates, under certain conditions, that application of such models is fair, transparent, and ethical, which consequently implies technical assessment of these models. This paper proposes and demonstrates an audit framework for technical assessment of RFMs and MLogRMs by focussing on model-, discrimination-, and transparency & explainability-related aspects. To measure these aspects 20 KPIs are proposed, which are paired to a traffic light risk assessment method. An open-source dataset is used to train a RFM and a MLogRM model and these KPIs are computed and compared with the traffic lights. The performance of popular explainability methods such as kernel- and tree-SHAP are assessed. The framework is expected to assist regulatory bodies in performing conformity assessments of binary classifiers and also benefits providers and users deploying such AI-systems to comply with the AIA.
Adaptive Representations of Sound for Automatic Insect Recognition
Venue: arXiv cs.SD 2022
Year: 2022

Insects are an integral part of our ecosystem. These often small and evasive animals have a big impact on their surroundings, providing a large part of the present biodiversity and pollination duties, forming the foundation of the food chain and many biological and ecological processes. Due to factors of human influence, population numbers and biodiversity have been rapidly declining with time. Monitoring this decline has become increasingly important for conservation measures to be effectively implemented. But monitoring methods are often invasive, time and resource intense, and prone to various biases. Many insect species produce characteristic mating sounds that can easily be detected and recorded without large cost or effort. Using deep learning methods, insect sounds from field recordings could be automatically detected and classified to monitor biodiversity and species distribution ranges. In this project, I implement this using existing datasets of insect sounds (Orthoptera and Cicadidae) and machine learning methods and evaluate their potential for acoustic insect monitoring. I compare the performance of the conventional spectrogram-based deep learning method against the new adaptive and waveform-based approach LEAF. The waveform-based frontend achieved significantly better classification performance than the Mel-spectrogram frontend by adapting its feature extraction parameters during training. This result is encouraging for future implementations of deep learning technology for automatic insect sound recognition, especially if larger datasets become available.
Towards Effective Cybercrime Intervention
Venue: arXiv cs.CR 2022
Year: 2022

Cybercrimes are on the rise, in part due to technological advancements, as well as increased avenues of exploitation. Sophisticated threat actors are leveraging on such advancements to execute their malicious intentions. The increase in cybercrimes is prevalent, and it seems unlikely that they can be easily eradicated. A more serious concern is that the community may come to accept the notion that this will become the trend. As such, the key question revolves around how we can reduce cybercrime in this evolving landscape. In our paper, we propose to build a systematic framework through the lens of a cyber threat actor. We explore the motivation factors behind the crimes and the crime stages of the threat actors. We then formulate intervention plans so as to discourage the act of committing malicious cyber activities and also aim to integrate ex-cyber offenders back into society.
Neural Langevin Dynamics: towards interpretable Neural Stochastic Differential Equations
Venue: arXiv cs.LG 2022
Year: 2022

Neural Stochastic Differential Equations (NSDE) have been trained as both Variational Autoencoders, and as GANs. However, the resulting Stochastic Differential Equations can be hard to interpret or analyse due to the generic nature of the drift and diffusion fields. By restricting our NSDE to be of the form of Langevin dynamics, and training it as a VAE, we obtain NSDEs that lend themselves to more elaborate analysis and to a wider range of visualisation techniques than a generic NSDE. More specifically, we obtain an energy landscape, the minima of which are in one-to-one correspondence with latent states underlying the used data. This not only allows us to detect states underlying the data dynamics in an unsupervised manner, but also to infer the distribution of time spent in each state according to the learned SDE. More in general, restricting an NSDE to Langevin dynamics enables the use of a large set of tools from computational molecular dynamics for the analysis of the obtained results.
A Reinforcement Learning Approach for Process Parameter Optimization in Additive Manufacturing
Venue: arXiv cs.LG 2022
Year: 2022

Process optimization for metal additive manufacturing (AM) is crucial to ensure repeatability, control microstructure, and minimize defects. Despite efforts to address this via the traditional design of experiments and statistical process mapping, there is limited insight on an on-the-fly optimization framework that can be integrated into a metal AM system. Additionally, most of these methods, being data-intensive, cannot be supported by a metal AM alloy or system due to budget restrictions. To tackle this issue, the article introduces a Reinforcement Learning (RL) methodology transformed into an optimization problem in the realm of metal AM. An off-policy RL framework based on Q-learning is proposed to find optimal laser power ($P$) - scan velocity ($v$) combinations with the objective of maintaining steady-state melt pool depth. For this, an experimentally validated Eagar-Tsai formulation is used to emulate the Laser-Directed Energy Deposition environment, where the laser operates as the agent across the $P-v$ space such that it maximizes rewards for a melt pool depth closer to the optimum. The culmination of the training process yields a Q-table where the state ($P,v$) with the highest Q-value corresponds to the optimized process parameter. The resultant melt pool depths and the mapping of Q-values to the $P-v$ space show congruence with experimental observations. The framework, therefore, provides a model-free approach to learning without any prior.
Fading in reflective and heavily shadowed industrial environments with large arrays
Venue: arXiv cs.IT 2022
Year: 2022

One of the use cases for 5G systems and beyond is ultra-reliability low-latency communication (URLLC). An enabling technology for URLLC is massive multiple-input multiple-output (MIMO), which can increase reliability due to improved user separation, array gain and the channel hardening effect. Measurements have been performed in an operating factory environment at 3.7 GHz with a co-located massive MIMO array and a unique randomly distributed array. Channel hardening can appear when the number of antennas is increased such that the variations of channel gain (small-scale fading) is decreased and it is here quantified. The cumulative distribution function (CDF) of the channel gains then becomes steeper and its tail is reduced. This CDF is modeled and the required fading margins are quantified. By deploying a distributed array, the large-scale power variations can also be reduced, further improving reliability. The large array in this rich scattering environment, creates a more reliable channel as it approaches an independent identically distributed (i.i.d.) complex Gaussian channel, indicating that one can rethink the system design in terms of e.g. channel coding and re-transmission strategies, in order to reduce latency. To conclude, massive MIMO is a highly interesting technology for reliable connectivity in reflective and heavily shadowed industrial environments.
Predicting Human Mobility via Self-supervised Disentanglement Learning
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural networks have recently achieved considerable improvements in learning human behavioral patterns and individual preferences from massive spatial-temporal trajectories data. However, most of the existing research concentrates on fusing different semantics underlying sequential trajectories for mobility pattern learning which, in turn, yields a narrow perspective on comprehending human intrinsic motions. In addition, the inherent sparsity and under-explored heterogeneous collaborative items pertaining to human check-ins hinder the potential exploitation of human diverse periodic regularities as well as common interests. Motivated by recent advances in disentanglement learning, in this study we propose a novel disentangled solution called SSDL for tackling the next POI prediction problem. SSDL primarily seeks to disentangle the potential time-invariant and time-varying factors into different latent spaces from massive trajectories data, providing an interpretable view to understand the intricate semantics underlying human diverse mobility representations. To address the data sparsity issue, we present two realistic trajectory augmentation approaches to enhance the understanding of both the human intrinsic periodicity and constantly-changing intents. In addition, we devise a POI-centric graph structure to explore heterogeneous collaborative signals underlying historical check-ins. Extensive experiments conducted on four real-world datasets demonstrate that our proposed SSDL significantly outperforms the state-of-the-art approaches -- for example, it yields up to 8.57% improvements on ACC@1.
Categorical Semantics for Model Comparison Games for Description Logics
Venue: arXiv cs.LO 2022
Year: 2022

A categorical approach to study model comparison games in terms of comonads was recently initiated by Abramsky et al. In this work, we analyse games that appear naturally in the context of description logics and supplement them with suitable game comonads. More precisely, we consider expressive sublogics of ALCSelfIbO, namely, the logics that extend ALC with any combination of inverses, nominals, safe boolean roles combinations, and Self operator. Our construction augments and modifies the so-called modal comonad by Abramsky and Shah. The approach that we took heavily relies on the use of relative comonads, which we leverage to encapsulate additional capabilities within the bisimulation games in a compositional manner.