Reviewerly/exHarmony huggingface dataset:
Pretraining the Noisy Channel Model for Task-Oriented Dialogue

Abstract Direct decoding for task-oriented dialogue is known to suffer from the explaining-away effect, manifested in models that prefer short and generic responses. Here we argue for the use of Bayes’ theorem to factorize the dialogue task into two models, the distribution of the context given the response, and the prior for the response itself. This approach, an instantiation of the noisy channel model, both mitigates the explaining-away effect and allows the principled incorporation of large pretrained models for the response prior. We present extensive experiments showing that a noisy channel model decodes better responses compared to direct decoding and that a two-stage pretraining strategy, employing both open-domain and task-oriented dialogue data, improves over randomly initialized models.
An improved extreme learning machine algorithm for transient electromagnetic nonlinear inversion

Transient electromagnetic method (TEM) inversion is significantly nonlinear. To eliminate the multicollinearity problem faced by the extreme learning machine (ELM) algorithm for TEM inversion, an improved ELM algorithm (F-ELM) based on fractal dimension technology is proposed. By reducing the dimension of the hidden layer output matrix (H) based on fractal dimension theory without losing the main statistical information, the proposed algorithm can not only guarantee the full column rank of the newly produced hidden layer output matrix (H′) but also enhance the training speed of the overall process. To prove the effectiveness of the F-ELM algorithm, a synthetic example and a field example using TEM inversion are established in this study. The experimental results illustrate that compared with the ordinary ELM algorithm and its variants, the proposed algorithm greatly reduces the computing time, improves the inversion accuracy and stability of the algorithm. Furthermore, it is also proven that the F-ELM algorithm is a very effective technique for TEM inversion.
GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularization

Deep learning is vulnerable to adversarial examples. Many defenses based on randomized neural networks have been proposed to solve the problem, but fail to achieve robustness against attacks using proxy gradients such as the Expectation over Transformation (EOT) attack. We investigate the effect of the adversarial attacks using proxy gradients on randomized neural networks and demonstrate that it highly relies on the directional distribution of the loss gradients of the randomized neural network. We show in particular that proxy gradients are less effective when the gradients are more scattered. To this end, we propose Gradient Diversity (GradDiv) regularizations that minimize the concentration of the gradients to build a robust randomized neural network. Our experiments on MNIST, CIFAR10, and STL10 show that our proposed GradDiv regularizations improve the adversarial robustness of randomized neural networks against a variety of state-of-the-art attack methods. Moreover, our method efficiently reduces the transferability among sample models of randomized neural networks.
Parallel hyper-heuristics for process engineering optimization

This paper presents the general framework of a parallel cooperative hyper-heuristic optimizer (PCHO) to solve systems of nonlinear algebraic equations with equality and inequality constraints. The algorithm comprises the classical metaheuristics called Genetic Algorithms, Simulated Annealing and Particle Swarm Optimization, whose parameters are adaptively chosen during the executions. A Master-Worker architecture was designed and implemented, where the Master processor ranks the solution candidates informed by the metaheuristics and immediately communicates the most promising candidate to update all Workers. Algorithmic performance was tested with general models, most of them corresponding to PSE process systems. The results confirmed the efficiency of the proposed approach since both online parameter retuning and parallel processing sped up the search.
A Small Target Detection Method Based on Deep Learning With Considerate Feature and Effectively Expanded Sample Size

As a basic task in the field of computer vision, target detection has been concerned by many researchers.The performance of target detection method is also directly related to the research in many advanced semantic fields.Current general target detection methods are not effective in small target detection, so this paper studies the problem of small target detection and proposes a small target detection method based on deep learning with considerate feature and effectively expanded sample size.Firstly, according to the characteristics of convolutional neural network, we improve the current deep network architecture which performs well in target detection, and introduce considerate multi-feature and multi-scale detection.Then, we transform the high-resolution images obtained on the Internet by combining two groups of sampling method, so that the feature distribution of the high-resolution target is closer to that of the low-resolution target, thus effectively expanding the training data set, solving the problem that small target data is difficult to be labeled and effectively avoiding overfitting.The results show the effectiveness of the improved method in small target detection.In addition, in view of the shortage of small target detection review literature, this paper gives a comprehensive and detailed introduction to the field of small target detection in terms of related work and future work.
Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graph

We present a novel two-layer hierarchical reinforcement learning approach equipped with a Goals Relational Graph (GRG) for tackling the partially observable goal-driven task, such as goal-driven visual navigation. Our GRG captures the underlying relations of all goals in the goal space through a Dirichlet-categorical process that facilitates: 1) the high-level network raising a sub-goal towards achieving a designated final goal; 2) the low-level network towards an optimal policy; and 3) the overall system generalizing unseen environments and goals. We evaluate our approach with two settings of partially observable goal-driven tasks — a grid-world domain and a robotic object search task. Our experimental results show that our approach exhibits superior generalization performance on both unseen environments and new goals <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Compcrypt–Lightweight ANS-Based Compression and Encryption

Compression is widely used in Internet applications to save communication time, bandwidth and storage. Recently invented by Jarek Duda asymmetric numeral system (ANS) offers an improved efficiency and a close to optimal compression. The ANS algorithm has been deployed by major IT companies such as Facebook, Google and Apple. Compression by itself does not provide any security (such as confidentiality or authentication of transmitted data). An obvious solution to this problem is an encryption of compressed bitstream. However, it requires two algorithms: one for compression and the other for encryption. In this work, we investigate natural properties of ANS that allow to incorporate authenticated encryption using as little cryptography as possible. We target low-level security communication and storage such as transmission of data from IoT devices/sensors. In particular, we propose three solutions for joint compression and encryption (compcrypt). The solutions offer different tradeoffs between security and efficiency assuming a slight compression deterioration. All of them use a pseudorandom bit generator (PRBG) based on lightweight stream ciphers. The first solution is close to original ANS and applies state jumps controlled by PRBG. The second one employs two copies of ANS, where compression is switched between the copies. The switch is controlled by a PRBG bit. The third compcrypt modifies the encoding function of ANS depending on PRBG bits. Security and efficiency of the proposed compcrypt algorithms are evaluated. The first compcrypt is the most efficient with a slight loss of compression quality. The second one consumes more storage but the loss of compression quality is negligible. The last compcrypt offers the best security but is the least efficient.
cuNH: Efficient GPU Implementations of Post-Quantum KEM NewHope

Post-quantum cryptography was proposed in the past years due to the foreseeable emergence of quantum computers that are able to break the conventional public key cryptosystems at acceptable costs. However, post-quantum schemes are usually less efficient than conventional ones, which makes them less practical in scenarios with limited resources or high concurrency. Server-side applications always feature multiple users, therefore requiring efficient execution of batch tasks. GPU is intrinsically well-suited to batch tasks owing to its SIMD/SIMT execution fashion, so it naturally helps to achieve high performance. However, a naive GPU-based implementation cannot make the best use of hardware resources of the GPU regardless of task loads. In this article, we propose SIMD parallelization paradigms for fine-grained GPU implementations and then apply them to a post-quantum key encapsulation algorithm called NewHope, where we carefully design every module, especially NTT and inverse NTT, to fit into the SIMD parallelization paradigms. In addition, we employ multi-streaming to improve performance in user's perspective. Finally, our evaluations are made on two testbeds with GPU accelerators NVIDIA GeForce MX150 and GeForce GTX 1650, respectively. The experimental results show that the fine-grained implementations save up to 98 percent latency at low task loads, and their throughputs increase by up to 86 percent at high task loads, when compared with the naive ones in kernel's perspective, and the multi-streaming implementations greatly reduce the latency overhead percentage at high task loads by up to 86 percent, when compared with the fine-grained implementation in user's perspective. Moreover, our fine-grained implementation and multi-streaming implementation are respectively 51.5 and 45.5 percent faster than Gupta et al.'s implementations when compared with it under reasonable assumptions. Furthermore, as lattice-based post-quantum schemes have similar operations, our proposal also easily applies to other lattice-based post-quantum schemes.
Towards Design Principles for User-Centric Explainable AI in Fraud Detection

Experts rely on fraud detection and decision support systems to analyze fraud cases, a growing problem in digital retailing and banking. With the advent of Artificial Intelligence (AI) for decision support, those experts face the black-box problem and lack trust in AI predictions for fraud. Such an issue has been tackled by employing Explainable AI (XAI) to provide experts with explained AI predictions through various explanation methods. However, fraud detection studies supported by XAI lack a user-centric perspective and discussion on how principles are deployed, both important requirements for experts to choose an appropriate explanation method. On the other hand, recent research in Information Systems (IS) and Human-Computer Interaction highlights the need for understanding user requirements to develop tailored design principles for decision support systems. In this research, we adopt a design science research methodology and IS theoretical lens to develop and evaluate design principles, which align fraud expert's tasks with explanation methods for Explainable AI decision support. We evaluate the utility of these principles using an information quality framework to interview experts in banking fraud, plus a simulation. The results show that the principles are an useful tool for designing decision support systems for fraud detection with embedded user-centric Explainable AI.
Learning Better Visual Dialog Agents with Pretrained Visual-Linguistic Representation

GuessWhat?! is a visual dialog guessing game which incorporates a Questioner agent that generates a sequence of questions, while an Oracle agent answers the respective questions about a target object in an image. Based on this dialog history between the Questioner and the Oracle, a Guesser agent makes a final guess of the target object. While previous work has focused on dialogue policy optimization and visual-linguistic information fusion, most work learns the vision-linguistic encoding for the three agents solely on the GuessWhat?! dataset without shared and prior knowledge of vision-linguistic representation. To bridge these gaps, this paper proposes new Oracle, Guesser and Questioner models that take advantage of a pretrained vision-linguistic model, VilBERT. For Oracle model, we introduce a two-way background/target fusion mechanism to understand both intra and inter-object questions. For Guesser model, we introduce a state-estimator that best utilizes VilBERT’s strength in single-turn referring expression comprehension. For the Questioner, we share the state-estimator from pretrained Guesser with Questioner to guide the question generator. Experimental results show that our proposed models outperform state-of-the-art models significantly by 7%, 10%, 12% for Oracle, Guesser and End-to-End Questioner respectively.
Answering Any-hop Open-domain Questions with Iterative Document Reranking

Existing approaches for open-domain question answering (QA) are typically designed for questions that require either single-hop or multi-hop reasoning, which make strong assumptions of the complexity of questions to be answered. Also, multi-step document retrieval often incurs higher number of relevant but non-supporting documents, which dampens the downstream noise-sensitive reader module for answer extraction. To address these challenges, we propose a unified QA framework to answer any-hop open-domain questions, which iteratively retrieves, reranks and filters documents, and adaptively determines when to stop the retrieval process. To improve the retrieval accuracy, we propose a graph-based reranking model that perform multi-document interaction as the core of our iterative reranking framework. Our method consistently achieves performance comparable to or better than the state-of-the-art on both single-hop and multi-hop open-domain QA datasets, including Natural Questions Open, SQuAD Open, and HotpotQA.
Discriminating features-based cost-sensitive approach for software defect prediction

Abstract Correlated quality metrics extracted from a source code repository can be utilized to design a model to automatically predict defects in a software system. It is obvious that the extracted metrics will result in a highly unbalanced data, since the number of defects in a good quality software system should be far less than the number of normal instances. It is also a fact that the selection of the best discriminating features significantly improves the robustness and accuracy of a prediction model. Therefore, the contribution of this paper is twofold, first it selects the best discriminating features that help in accurately predicting a defect in a software component. Secondly, a cost-sensitive logistic regression and decision tree ensemble-based prediction models are applied to the best discriminating features for precisely predicting a defect in a software component. The proposed models are compared with the most recent schemes in the literature in terms of accuracy, area under the curve, and recall. The models are evaluated using 11 datasets and it is evident from the results and analysis that the performance of the proposed prediction models outperforms the schemes in the literature.
Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequences

Multi-object tracking from RGB-D video sequences is a challenging problem due to the combination of changing viewpoints, motion, and occlusions over time. We observe that having the complete geometry of objects aids in their tracking, and thus propose to jointly infer the complete geometry of objects as well as track them, for rigidly moving objects over time. Our key insight is that inferring the complete geometry of the objects significantly helps in tracking. By hallucinating unseen regions of objects, we can obtain additional correspondences between the same instance, thus providing robust tracking even under strong change of appearance. From a sequence of RGB-D frames, we detect objects in each frame and learn to predict their complete object geometry as well as a dense correspondence mapping into a canonical space. This allows us to derive 6DoF poses for the objects in each frame, along with their correspondence between frames, providing robust object tracking across the RGB-D sequence. Experiments on both synthetic and real-world RGB-D data demonstrate that we achieve state-of-the-art performance on dynamic object tracking. Furthermore, we show that our object completion significantly helps tracking, providing an improvement of 6.5% in mean MOTA.
Fully convolutional open set segmentation

In traditional semantic segmentation, knowing about all existing classes is essential to yield effective results with the majority of existing approaches. However, these methods trained in a Closed Set of classes fail when new classes are found in the test phase, not being able to recognize that an unseen class has been fed. This means that they are not suitable for Open Set scenarios, which are very common in real-world computer vision and remote sensing applications. In this paper, we discuss the limitations of Closed Set segmentation and propose two fully convolutional approaches to effectively address Open Set semantic segmentation: OpenFCN and OpenPCS. OpenFCN is based on the well-known OpenMax algorithm, configuring a new application of this approach in segmentation settings. OpenPCS is a fully novel approach based on feature-space from DNN activations that serve as features for computing PCA and multi-variate gaussian likelihood in a lower dimensional space. In addition to OpenPCS and aiming to reduce the RAM memory requirements of the methodology, we also propose a slight variation of the method (OpenIPCS) that uses an iteractive version of PCA able to be trained in small batches. Experiments were conducted on the well-known ISPRS Vaihingen/Potsdam and the 2018 IEEE GRSS Data Fusion Challenge datasets. OpenFCN showed little-to-no improvement when compared to the simpler and much more time efficient SoftMax thresholding, while being some orders of magnitude slower. OpenPCS achieved promising results in almost all experiments by overcoming both OpenFCN and SoftMax thresholding. OpenPCS is also a reasonable compromise between the runtime performances of the extremely fast SoftMax thresholding and the extremely slow OpenFCN, being able to run close to real-time. Experiments also indicate that OpenPCS is effective, robust and suitable for Open Set segmentation, being able to improve the recognition of unknown class pixels without reducing the accuracy on the known class pixels. We also tested the scenario of hiding multiple known classes to simulate multimodal unknowns, resulting in an even larger gap between OpenPCS/OpenIPCS and both SoftMax thresholding and OpenFCN, implying that gaussian modeling is more robust to settings with greater openness.
Model learning with personalized interpretability estimation (ML-PIE)

High-stakes applications require AI-generated models to be interpretable. Current algorithms for the synthesis of potentially interpretable models rely on objectives or regularization terms that represent interpretability only coarsely (e.g., model size) and are not designed for a specific user. Yet, interpretability is intrinsically subjective. In this paper, we propose an approach for the synthesis of models that are tailored to the user by enabling the user to steer the model synthesis process according to her or his preferences. We use a bi-objective evolutionary algorithm to synthesize models with trade-offs between accuracy and a user-specific notion of interpretability. The latter is estimated by a neural network that is trained concurrently to the evolution using the feedback of the user, which is collected using uncertainty-based active learning. To maximize usability, the user is only asked to tell, given two models at the time, which one is less complex. With experiments on two real-world datasets involving 61 participants, we find that our approach is capable of learning estimations of interpretability that can be very different for different users. Moreover, the users tend to prefer models found using the proposed approach over models found using non-personalized interpretability indices.
Automatic recommendation of feature selection algorithms based on dataset characteristics

Feature selection in real-world data mining problems is essential to make the learning task efficient and more accurate. Identifying the best feature selection algorithm, among the many available, is a complex activity that still relies heavily on human experts or some random trial-and-error procedure. Thus, the automated machine learning community has taken some steps towards the automation of this process. In this paper, we address the metalearning challenge of recommending feature selection algorithms by proposing a novel meta-feature engineering model. Our model considers a broad collection of meta-features that enable the study of the relationship between the dataset properties and the feature selection algorithm performance in terms of several criteria. We arrange the input meta-features into eight categories: (i) simple, (ii) statistical, (iii) information-theoretical, (iv) complexity, (v) landmarking, (vi) based on symbolic models, (vii) based on images, and (viii) based on complex networks (graphs). The target meta-features emerge from a multi-criteria performance measure, based on five individual performance indexes, that assesses feature selection methods grounded in information, distance, dependence, consistency, and precision measures. We evaluate our proposal using a recently developed framework that extracts the input meta-features from 213 benchmark datasets, and ranks the assessed feature selection algorithms, to fill in the target meta-features in meta-bases. This evaluation uses five state-of-the-art classification methods to induce recommendation models from meta-bases: C4.5, Random Forest, XGBoost, ANN, and SVM. The results showed that it is possible to reach an average accuracy of up to 90% applying our meta-feature engineering model. This work is the first to use an extensive empirical evaluation to provide a careful discussion of the strengths and limitations of more than 160 meta-features. These meta-features, while designed to aid the task of feature selection algorithm recommendation, can readily be employed in other metalearning scenarios. Therefore, we believe our findings are a valuable contribution to the fields of automated machine learning and data mining, as well as to the feature extraction and pattern recognition communities.
The Case for AI Based Web3 Reputation Systems

Initiatives such as blockchains and decentralized storage networks are pushing for a decentralized Web3 to replace the current architecture. At the core of Web3 are network resource sharing services, which allow anyone to sell spare network capacity in return for rewards. These services require a way to establish trust, as parties are potentially malicious. This can be achieved by reputation systems. In this paper we make the case for using deep reinforcement learning in Web3 reputation calculation. More specifically, we propose a model which allows for decentralized calculation of scores with high personalization for the user.