OpenRaiser/Intern-Atlas huggingface dataset:
Frequency-Shift Chirp Spread Spectrum Communications with Index Modulation
Venue: arXiv cs.IT 2021
Year: 2021

This paper introduces a novel frequency-shift chirp spread spectrum (FSCSS) system with index modulation (IM). By using combinations of orthogonal chirp signals for message representation, the proposed FSCSS-IM system is very flexible to design and can achieve much higher data rates than the conventional FSCSS system under the same bandwidth. The paper presents optimal detection algorithms, both coherently and non-coherently, for the proposed FSCSS-IM system. Furthermore, a low-complexity non-coherent detection algorithm is also developed to reduce the computational complexity of the receiver, which is shown to achieve near-optimal performance. Results are presented to demonstrate that the proposed system, while enabling much higher data rates, enjoys similar bit-error performance as that of the conventional FSCSS system.
Efficient Retrieval Augmented Generation from Unstructured Knowledge for Task-Oriented Dialog
Venue: arXiv cs.CL 2021
Year: 2021

This paper summarizes our work on the first track of the ninth Dialog System Technology Challenge (DSTC 9), "Beyond Domain APIs: Task-oriented Conversational Modeling with Unstructured Knowledge Access". The goal of the task is to generate responses to user turns in a task-oriented dialog that require knowledge from unstructured documents. The task is divided into three subtasks: detection, selection and generation. In order to be compute efficient, we formulate the selection problem in terms of hierarchical classification steps. We achieve our best results with this model. Alternatively, we employ siamese sequence embedding models, referred to as Dense Knowledge Retrieval, to retrieve relevant documents. This method further reduces the computation time by a factor of more than 100x at the cost of degradation in R@1 of 5-6% compared to the first model. Then for either approach, we use Retrieval Augmented Generation to generate responses based on multiple selected snippets and we show how the method can be used to fine-tune trained embeddings.
Large Scale Long-tailed Product Recognition System at Alibaba
Venue: arXiv cs.CV 2021
Year: 2021

A practical large scale product recognition system suffers from the phenomenon of long-tailed imbalanced training data under the E-commercial circumstance at Alibaba. Besides product images at Alibaba, plenty of image related side information (e.g. title, tags) reveal rich semantic information about images. Prior works mainly focus on addressing the long tail problem in visual perspective only, but lack of consideration of leveraging the side information. In this paper, we present a novel side information based large scale visual recognition co-training~(SICoT) system to deal with the long tail problem by leveraging the image related side information. In the proposed co-training system, we firstly introduce a bilinear word attention module aiming to construct a semantic embedding over the noisy side information. A visual feature and semantic embedding co-training scheme is then designed to transfer knowledge from classes with abundant training data (head classes) to classes with few training data (tail classes) in an end-to-end fashion. Extensive experiments on four challenging large scale datasets, whose numbers of classes range from one thousand to one million, demonstrate the scalable effectiveness of the proposed SICoT system in alleviating the long tail problem. In the visual search platform Pailitao\footnote{http://www.pailitao.com} at Alibaba, we settle a practical large scale product recognition application driven by the proposed SICoT system, and achieve a significant gain of unique visitor~(UV) conversion rate.
CorrDetector: A Framework for Structural Corrosion Detection from Drone Images using Ensemble Deep Learning
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose a new technique that applies automated image analysis in the area of structural corrosion monitoring and demonstrate improved efficacy compared to existing approaches. Structural corrosion monitoring is the initial step of the risk-based maintenance philosophy and depends on an engineer's assessment regarding the risk of building failure balanced against the fiscal cost of maintenance. This introduces the opportunity for human error which is further complicated when restricted to assessment using drone captured images for those areas not reachable by humans due to many background noises. The importance of this problem has promoted an active research community aiming to support the engineer through the use of artificial intelligence (AI) image analysis for corrosion detection. In this paper, we advance this area of research with the development of a framework, CorrDetector. CorrDetector uses a novel ensemble deep learning approach underpinned by convolutional neural networks (CNNs) for structural identification and corrosion feature extraction. We provide an empirical evaluation using real-world images of a complicated structure (e.g. telecommunication tower) captured by drones, a typical scenario for engineers. Our study demonstrates that the ensemble approach of \model significantly outperforms the state-of-the-art in terms of classification accuracy.
Synchronous Forwarders
Venue: arXiv cs.PL 2021
Year: 2021

Session types are types for specifying protocols that processes must follow when communicating with each other. Session types are in a propositions-as-types correspondence with linear logic. Previous work has shown that a multiparty session type, a generalisation of session types to protocols of two or more parties, can be modelled as a proof of coherence, a generalisation of linear logic duality. And, protocols expressed as coherence can be simulated by arbiters, processes that act as a middleware by forwarding messages according to the given protocol. In this paper, we generalise the concept of arbiter to that of synchronous forwarder, that is a processes that implements the behaviour of an arbiter in several different ways. In a propositions-as-types fashion, synchronous forwarders form a logic equipped with cut elimination which is a special restriction of classical linear logic. Our main result shows that synchronous forwarders are a characterisation of coherence, i.e., coherence proofs can be transformed into synchronous forwarders and, viceversa, every synchronous forwarder corresponds to a coherence proofs.
Transparency to hybrid open access through publisher-provided metadata: An article-level study of Elsevier
Venue: arXiv cs.DL 2021
Year: 2021

With the growth of open access (OA), the financial flows in scholarly journal publishing have become increasingly complex, but comprehensive data and transparency into these flows are still lacking. The opaqueness is especially concerning for hybrid OA, where subscription-based journals publish individual articles as OA if an optional fee is paid. This study addresses the lack of transparency by leveraging Elsevier article metadata and provides the first publisher-level study of hybrid OA uptake and invoicing. Our results show that Elsevier's hybrid OA uptake has grown steadily but slowly from 2015-2019, doubling the number of hybrid OA articles published per year and increasing the share of OA articles in Elsevier's hybrid journals from 2.6% to 3.7% of all articles. Further, we find that most hybrid OA articles were invoiced directly to authors, followed by articles invoiced through agreements with research funders, institutions, or consortia, with only a few funding bodies driving hybrid OA uptake. As such, our findings point to the role of publishing agreements and OA policies in hybrid OA publishing. Our results further demonstrate the value of publisher-provided metadata to improve the transparency in scholarly publishing by linking invoicing data to bibliometrics.
Avaddon ransomware: an in-depth analysis and decryption of infected systems
Venue: arXiv cs.CR 2021
Year: 2021

The commoditization of Malware-as-a-Service (MaaS) allows criminals to obtain financial benefits at a low risk and with little technical background. One such popular product in the underground economy is ransomware. In ransomware attacks, data from infected systems is held hostage (encrypted) until a fee is paid to the criminals. This modus operandi disrupts legitimate businesses, which may become unavailable until the data is restored. A recent blackmailing strategy adopted by criminals is to leak data online from the infected systems if the ransom is not paid. Besides reputational damage, data leakage might produce further economical losses due to fines imposed by data protection laws. Thus, research on prevention and recovery measures to mitigate the impact of such attacks is needed to adapt existing countermeasures to new strains. In this work, we perform an in-depth analysis of Avaddon, a ransomware offered in the underground economy as an affiliate program business. This has infected and leaked data from at least 23 organizations. Additionally, it runs Distributed Denial-of-Service (DDoS) attacks against victims that do not pay the ransom. We first provide an analysis of the criminal business model from the underground economy. Then, we identify and describe its technical capabilities. We provide empirical evidence of links between this variant and a previous family, suggesting that the same group was behind the development and, possibly, the operation of both campaigns. Finally, we describe a method to decrypt files encrypted with Avaddon in real time. We implement and test the decryptor in a tool that can recover the encrypted data from an infected system, thus mitigating the damage caused by the ransomware. The tool is released open-source so it can be incorporated in existing Antivirus engines.
Smart non-intrusive appliance identification using a novel local power histogramming descriptor with an improved k-nearest neighbors classifier
Venue: arXiv cs.CY 2021
Year: 2021

Non-intrusive load monitoring (NILM) is a key cost-effective technology for monitoring power consumption and contributing to several challenges encountered when transiting to an efficient, sustainable, and competitive energy efficiency environment. This paper proposes a smart NILM system based on a novel local power histogramming (LPH) descriptor, in which appliance power signals are transformed into 2D space and short histograms are extracted to represent each device. Specifically, short local histograms are drawn to represent individual appliance consumption signatures and robustly extract appliance-level data from the aggregated power signal. Furthermore, an improved k-nearest neighbors (IKNN) algorithm is presented to reduce the learning computation time and improve the classification performance. This results in highly improving the discrimination ability between appliances belonging to distinct categories. A deep evaluation of the proposed LPH-IKNN based solution is investigated under different data sets, in which the proposed scheme leads to promising performance. An accuracy of up to 99.65% and 98.51% has been achieved on GREEND and UK-DALE data sets, respectively. While an accuracy of more than 96% has been attained on both WHITED and PLAID data sets. This proves the validity of using 2D descriptors to accurately identify appliances and create new perspectives for the NILM problem.
Learning How to Search: Generating Effective Test Cases Through Adaptive Fitness Function Selection
Venue: arXiv cs.SE 2021
Year: 2021

Search-based test generation is guided by feedback from one or more fitness functions - scoring functions that judge solution optimality. Choosing informative fitness functions is crucial to meeting the goals of a tester. Unfortunately, many goals - such as forcing the class-under-test to throw exceptions, increasing test suite diversity, and attaining Strong Mutation Coverage - do not have effective fitness function formulations. We propose that meeting such goals requires treating fitness function identification as a secondary optimization step. An adaptive algorithm that can vary the selection of fitness functions could adjust its selection throughout the generation process to maximize goal attainment, based on the current population of test suites. To test this hypothesis, we have implemented two reinforcement learning algorithms in the EvoSuite unit test generation framework, and used these algorithms to dynamically set the fitness functions used during generation for the three goals identified above. We have evaluated our framework, EvoSuiteFIT, on a set of Java case examples. EvoSuiteFIT techniques attain significant improvements for two of the three goals, and show limited improvements on the third when the number of generations of evolution is fixed. Additionally, for two of the three goals, EvoSuiteFIT detects faults missed by the other techniques. The ability to adjust fitness functions allows strategic choices that efficiently produce more effective test suites, and examining these choices offers insight into how to attain our testing goals. We find that adaptive fitness function selection is a powerful technique to apply when an effective fitness function does not already exist for achieving a testing goal.
The Factory Must Grow: Automation in Factorio
Venue: arXiv cs.AI 2021
Year: 2021

Efficient optimization of resources is paramount to success in many problems faced today. In the field of operational research the efficient scheduling of employees; packing of vans; routing of vehicles; logistics of airlines and transport of materials can be the difference between emission reduction or excess, profits or losses and feasibility or unworkable solutions. The video game Factorio, by Wube Software, has a myriad of problems which are analogous to such real-world problems, and is a useful simulator for developing solutions for these problems. In this paper we define the logistic transport belt problem and define mathematical integer programming model of it. We developed an interface to allow optimizers in any programming language to interact with Factorio, and we provide an initial benchmark of logistic transport belt problems. We present results for Simulated Annealing, quick Genetic Programming and Evolutionary Reinforcement Learning, three different meta-heuristic techniques to optimize this novel problem.
On Theory-training Neural Networks to Infer the Solution of Highly Coupled Differential Equations
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks are transforming fields ranging from computer vision to computational medicine, and we recently extended their application to the field of phase-change heat transfer by introducing theory-trained neural networks (TTNs) for a solidification problem \cite{TTN}. Here, we present general, in-depth, and empirical insights into theory-training networks for learning the solution of highly coupled differential equations. We analyze the deteriorating effects of the oscillating loss on the ability of a network to satisfy the equations at the training data points, measured by the final training loss, and on the accuracy of the inferred solution. We introduce a theory-training technique that, by leveraging regularization, eliminates those oscillations, decreases the final training loss, and improves the accuracy of the inferred solution, with no additional computational cost. Then, we present guidelines that allow a systematic search for the network that has the optimal training time and inference accuracy for a given set of equations; following these guidelines can reduce the number of tedious training iterations in that search. Finally, a comparison between theory-training and the rival, conventional method of solving differential equations using discretization attests to the advantages of theory-training not being necessarily limited to high-dimensional sets of equations. The comparison also reveals a limitation of the current theory-training framework that may limit its application in domains where extreme accuracies are necessary.
Dynamic Neural Networks: A Survey
Venue: arXiv cs.CV 2021
Year: 2021

Dynamic neural network is an emerging research topic in deep learning. Compared to static models which have fixed computational graphs and parameters at the inference stage, dynamic networks can adapt their structures or parameters to different inputs, leading to notable advantages in terms of accuracy, computational efficiency, adaptiveness, etc. In this survey, we comprehensively review this rapidly developing area by dividing dynamic networks into three main categories: 1) instance-wise dynamic models that process each instance with data-dependent architectures or parameters; 2) spatial-wise dynamic networks that conduct adaptive computation with respect to different spatial locations of image data and 3) temporal-wise dynamic models that perform adaptive inference along the temporal dimension for sequential data such as videos and texts. The important research problems of dynamic networks, e.g., architecture design, decision making scheme, optimization technique and applications, are reviewed systematically. Finally, we discuss the open problems in this field together with interesting future research directions.
rl_reach: Reproducible Reinforcement Learning Experiments for Robotic Reaching Tasks
Venue: arXiv cs.LG 2021
Year: 2021

Training reinforcement learning agents at solving a given task is highly dependent on identifying optimal sets of hyperparameters and selecting suitable environment input / output configurations. This tedious process could be eased with a straightforward toolbox allowing its user to quickly compare different training parameter sets. We present rl_reach, a self-contained, open-source and easy-to-use software package designed to run reproducible reinforcement learning experiments for customisable robotic reaching tasks. rl_reach packs together training environments, agents, hyperparameter optimisation tools and policy evaluation scripts, allowing its users to quickly investigate and identify optimal training configurations. rl_reach is publicly available at this URL: https://github.com/PierreExeter/rl_reach.
More Is More -- Narrowing the Generalization Gap by Adding Classification Heads
Venue: arXiv cs.LG 2021
Year: 2021

Overfit is a fundamental problem in machine learning in general, and in deep learning in particular. In order to reduce overfit and improve generalization in the classification of images, some employ invariance to a group of transformations, such as rotations and reflections. However, since not all objects exhibit necessarily the same invariance, it seems desirable to allow the network to learn the useful level of invariance from the data. To this end, motivated by self-supervision, we introduce an architecture enhancement for existing neural network models based on input transformations, termed 'TransNet', together with a training algorithm suitable for it. Our model can be employed during training time only and then pruned for prediction, resulting in an equivalent architecture to the base model. Thus pruned, we show that our model improves performance on various data-sets while exhibiting improved generalization, which is achieved in turn by enforcing soft invariance on the convolutional kernels of the last layer in the base model. Theoretical analysis is provided to support the proposed method.
Robust Motion In-betweening
Venue: arXiv cs.CV 2021
Year: 2021

In this work we present a novel, robust transition generation technique that can serve as a new tool for 3D animators, based on adversarial recurrent neural networks. The system synthesizes high-quality motions that use temporally-sparse keyframes as animation constraints. This is reminiscent of the job of in-betweening in traditional animation pipelines, in which an animator draws motion frames between provided keyframes. We first show that a state-of-the-art motion prediction model cannot be easily converted into a robust transition generator when only adding conditioning information about future keyframes. To solve this problem, we then propose two novel additive embedding modifiers that are applied at each timestep to latent representations encoded inside the network's architecture. One modifier is a time-to-arrival embedding that allows variations of the transition length with a single model. The other is a scheduled target noise vector that allows the system to be robust to target distortions and to sample different transitions given fixed keyframes. To qualitatively evaluate our method, we present a custom MotionBuilder plugin that uses our trained model to perform in-betweening in production scenarios. To quantitatively evaluate performance on transitions and generalizations to longer time horizons, we present well-defined in-betweening benchmarks on a subset of the widely used Human3.6M dataset and on LaFAN1, a novel high quality motion capture dataset that is more appropriate for transition generation. We are releasing this new dataset along with this work, with accompanying code for reproducing our baseline results.
Hallmarks of Human-Machine Collaboration: A framework for assessment in the DARPA Communicating with Computers Program
Venue: arXiv cs.HC 2021
Year: 2021

There is a growing desire to create computer systems that can communicate effectively to collaborate with humans on complex, open-ended activities. Assessing these systems presents significant challenges. We describe a framework for evaluating systems engaged in open-ended complex scenarios where evaluators do not have the luxury of comparing performance to a single right answer. This framework has been used to evaluate human-machine creative collaborations across story and music generation, interactive block building, and exploration of molecular mechanisms in cancer. These activities are fundamentally different from the more constrained tasks performed by most contemporary personal assistants as they are generally open-ended, with no single correct solution, and often no obvious completion criteria. We identified the Key Properties that must be exhibited by successful systems. From there we identified "Hallmarks" of success -- capabilities and features that evaluators can observe that would be indicative of progress toward achieving a Key Property. In addition to being a framework for assessment, the Key Properties and Hallmarks are intended to serve as goals in guiding research direction.
Semantic Borrowing for Generalized Zero-Shot Learning
Venue: arXiv cs.LG 2021
Year: 2021

Generalized zero-shot learning (GZSL) is one of the most realistic but challenging problems due to the partiality of the classifier to supervised classes, especially under the class-inductive instance-inductive (CIII) training setting, where testing data are not available. Instance-borrowing methods and synthesizing methods solve it to some extent with the help of testing semantics, but therefore neither can be used under CIII. Besides, the latter require the training process of a classifier after generating examples. In contrast, a novel non-transductive regularization under CIII called Semantic Borrowing (SB) for improving GZSL methods with compatibility metric learning is proposed in this paper, which not only can be used for training linear models, but also nonlinear ones such as artificial neural networks. This regularization item in the loss function borrows similar semantics in the training set, so that the classifier can model the relationship between the semantics of zero-shot and supervised classes more accurately during training. In practice, the information of semantics of unknown classes would not be available for training while this approach does NOT need it. Extensive experiments on GZSL benchmark datasets show that SB can reduce the partiality of the classifier to supervised classes and improve the performance of generalized zero-shot classification, surpassing inductive GZSL state of the arts.