OpenRaiser/Intern-Atlas huggingface dataset:
Automatic Generation of Blockchain Agri-food Traceability Systems
Venue: arXiv cs.SE 2021
Year: 2021

Supply chain management, product provenance and quality certification are among the first and most popular applications of blockchain technology, due to the inherent trust and inalterability provided by the blockchain technology. However, the proposed supply chain management systems based on blockchain and smart contract technology tend to be specific to the particular production and production process. In this paper we present a general-purpose approach for the agri-food supply chain management, proposing a system that can be configured for most agri-food productions. The primary purpose is to provide a methodology to facilitate and make more efficient the development of such applications. It is based on general smart contracts and apps interacting with the same smart contracts, which are configured, starting from the description of the specific system to be managed, given using json files. A case study on olive oil production is described, to show how our approach works.
EventGraD: Event-Triggered Communication in Parallel Machine Learning
Venue: arXiv cs.LG 2021
Year: 2021

Communication in parallel systems imposes significant overhead which often turns out to be a bottleneck in parallel machine learning. To relieve some of this overhead, in this paper, we present EventGraD - an algorithm with event-triggered communication for stochastic gradient descent in parallel machine learning. The main idea of this algorithm is to modify the requirement of communication at every iteration in standard implementations of stochastic gradient descent in parallel machine learning to communicating only when necessary at certain iterations. We provide theoretical analysis of convergence of our proposed algorithm. We also implement the proposed algorithm for data-parallel training of a popular residual neural network used for training the CIFAR-10 dataset and show that EventGraD can reduce the communication load by up to 60% while retaining the same level of accuracy. In addition, EventGraD can be combined with other approaches such as Top-K sparsification to decrease communication further while maintaining accuracy.
Private Cross-Silo Federated Learning for Extracting Vaccine Adverse Event Mentions
Venue: arXiv cs.LG 2021
Year: 2021

Federated Learning (FL) is quickly becoming a goto distributed training paradigm for users to jointly train a global model without physically sharing their data. Users can indirectly contribute to, and directly benefit from a much larger aggregate data corpus used to train the global model. However, literature on successful application of FL in real-world problem settings is somewhat sparse. In this paper, we describe our experience applying a FL based solution to the Named Entity Recognition (NER) task for an adverse event detection application in the context of mass scale vaccination programs. We present a comprehensive empirical analysis of various dimensions of benefits gained with FL based training. Furthermore, we investigate effects of tighter Differential Privacy (DP) constraints in highly sensitive settings where federation users must enforce Local DP to ensure strict privacy guarantees. We show that local DP can severely cripple the global model's prediction accuracy, thus dis-incentivizing users from participating in the federation. In response, we demonstrate how recent innovation on personalization methods can help significantly recover the lost accuracy. We focus our analysis on the Federated Fine-Tuning algorithm, FedFT, and prove that it is not PAC Identifiable, thus making it even more attractive for FL-based training.
Discovery of Physics and Characterization of Microstructure from Data with Bayesian Hidden Physics Models
Venue: arXiv cs.LG 2021
Year: 2021

There has been a surge in the interest of using machine learning techniques to assist in the scientific process of formulating knowledge to explain observational data. We demonstrate the use of Bayesian Hidden Physics Models to first uncover the physics governing the propagation of acoustic impulses in metallic specimens using data obtained from a pristine sample. We then use the learned physics to characterize the microstructure of a separate specimen with a surface-breaking crack flaw. Remarkably, we find that the physics learned from the first specimen allows us to understand the backscattering observed in the latter sample, a qualitative feature that is wholly absent from the specimen from which the physics were inferred. The backscattering is explained through inhomogeneities of a latent spatial field that can be recognized as the speed of sound in the media.
Safe Sampling-Based Air-Ground Rendezvous Algorithm for Complex Urban Environments
Venue: arXiv cs.RO 2021
Year: 2021

Demand for fast and economical parcel deliveries in urban environments has risen considerably in recent years. A framework envisions efficient last-mile delivery in urban environments by leveraging a network of ride-sharing vehicles, where Unmanned Aerial Systems (UASs) drop packages on said vehicles, which then cover the majority of the distance before final aerial delivery. Notably, we consider the problem of planning a rendezvous path for the UAS to reach a human driver, who may choose between N possible paths and has uncertain behavior, while meeting strict safety constraints. The long planning horizon and safety constraints require robust heuristics that combine learning and optimal control using Gaussian Process Regression, sampling-based optimization, and Model Predictive Control. The resulting algorithm is computationally efficient and shown to be effective in a variety of qualitative scenarios.
A Distributed Optimisation Framework Combining Natural Gradient with Hessian-Free for Discriminative Sequence Training
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents a novel natural gradient and Hessian-free (NGHF) optimisation framework for neural network training that can operate efficiently in a distributed manner. It relies on the linear conjugate gradient (CG) algorithm to combine the natural gradient (NG) method with local curvature information from Hessian-free (HF) or other second-order methods. A solution to a numerical issue in CG allows effective parameter updates to be generated with far fewer CG iterations than usually used (e.g. 5-8 instead of 200). This work also presents a novel preconditioning approach to improve the progress made by individual CG iterations for models with shared parameters. Although applicable to other training losses and model structures, NGHF is investigated in this paper for lattice-based discriminative sequence training for hybrid hidden Markov model acoustic models using a standard recurrent neural network, long short-term memory, and time delay neural network models for output probability calculation. Automatic speech recognition experiments are reported on the multi-genre broadcast data set for a range of different acoustic model types. These experiments show that NGHF achieves larger word error rate reductions than standard stochastic gradient descent or Adam, while requiring orders of magnitude fewer parameter updates.
First-Order Sketch Conditions and Constraints -- A Category Independent Approach
Venue: arXiv cs.LO 2021
Year: 2021

Generalizing different variants of "graph conditions and constraints" as well as "universal constraints" and "negative universal constraints" in the Diagram Predicate Framework (DPF), we introduce for arbitrary categories $\mathbf{Cxt}$ and "statement" functors $\mathtt{Stm}:\mathbf{Cxt}\to\mathbf{Set}$ general first-order sketch conditions and constraints. Sketches are used in DPF to formalize different kinds of diagrammatic software models. We discuss the use of sketch constraints for describing the syntactic structure of sketches. We outline the use of sketch constraints to deduce knowledge implicitly given in a sketch as well as procedures to deduce sketch constraints from given sketch constraints. We use the simple but paradigmatic modeling formalism "Category Theory" as running example.
To not miss the forest for the trees -- a holistic approach for explaining missing answers over nested data (extended version)
Venue: arXiv cs.DB 2021
Year: 2021

Query-based explanations for missing answers identify which operators of a query are responsible for the failure to return a missing answer of interest. This type of explanations has proven to be useful in a variety of contexts including debugging of complex analytical queries. Such queries are frequent in big data systems such as Apache Spark. We present a novel approach for producing query-based explanations. Our approach is the first to support nested data and to consider operators that modify the schema and structure of the data (e.g., nesting and projections) as potential causes of missing answers. To efficiently compute explanations, we propose a heuristic algorithm that applies two novel techniques: (i) reasoning about multiple schema alternatives for a query and (ii) re-validating at each step whether an intermediate result can contribute to the missing answer. Using an implementation of our approach on Spark, we demonstrate that it is the first to scale to large datasets and that it often finds explanations that existing techniques fail to identify.
Application of HL7 FHIR in a Microservice Architecture for Patient Navigation on Registration and Appointments
Venue: arXiv cs.SE 2021
Year: 2021

Electronic Health Record Systems (EHR-S) are commonly developed in monolithic architectures. This architectural style presents greater complexity and demands more effort when we think of interoperability. A solution proposal is the creation of Microservices that use HL7 FHIR as an interoperability strategy. In this sense, it is presented the development of a prototype, based on a microservices architecture, to act in a real scenario of Patient Navigation (PN). The problem was subdivided into 3 steps: definition of architecture, development and construction of an interface to simulate the role of the navigator. The Patient and Appointment microservices are capable of synchronous communication to query and record information. In general, the implemented architectural style not only isolates information domains but can receive data from multiple sources while maintaining essential functionality. This type of approach plays a crucial role in a hospital environment, specifically in PN, highlighting the importance of the standard and expanding the possibilities for further research to be conducted.
Learning Defense Transformers for Counterattacking Adversarial Examples
Venue: arXiv cs.LG 2021
Year: 2021

Deep neural networks (DNNs) are vulnerable to adversarial examples with small perturbations. Adversarial defense thus has been an important means which improves the robustness of DNNs by defending against adversarial examples. Existing defense methods focus on some specific types of adversarial examples and may fail to defend well in real-world applications. In practice, we may face many types of attacks where the exact type of adversarial examples in real-world applications can be even unknown. In this paper, motivated by that adversarial examples are more likely to appear near the classification boundary, we study adversarial examples from a new perspective that whether we can defend against adversarial examples by pulling them back to the original clean distribution. We theoretically and empirically verify the existence of defense affine transformations that restore adversarial examples. Relying on this, we learn a defense transformer to counterattack the adversarial examples by parameterizing the affine transformations and exploiting the boundary information of DNNs. Extensive experiments on both toy and real-world datasets demonstrate the effectiveness and generalization of our defense transformer.
DeepGroup: Representation Learning for Group Recommendation with Implicit Feedback
Venue: arXiv cs.AI 2021
Year: 2021

Group recommender systems facilitate group decision making for a set of individuals (e.g., a group of friends, a team, a corporation, etc.). Many of these systems, however, either assume that (i) user preferences can be elicited (or inferred) and then aggregated into group preferences or (ii) group preferences are partially observed/elicited. We focus on making recommendations for a new group of users whose preferences are unknown, but we are given the decisions/choices of other groups. By formulating this problem as group recommendation from group implicit feedback, we focus on two of its practical instances: group decision prediction and reverse social choice. Given a set of groups and their observed decisions, group decision prediction intends to predict the decision of a new group of users, whereas reverse social choice aims to infer the preferences of those users involved in observed group decisions. These two problems are of interest to not only group recommendation, but also to personal privacy when the users intend to conceal their personal preferences but have participated in group decisions. To tackle these two problems, we propose and study DeepGroup -- a deep learning approach for group recommendation with group implicit data. We empirically assess the predictive power of DeepGroup on various real-world datasets, group conditions (e.g., homophily or heterophily), and group decision (or voting) rules. Our extensive experiments not only demonstrate the efficacy of DeepGroup, but also shed light on the privacy-leakage concerns of some decision making processes.
Representative volume elements for matrix-inclusion composites -- a computational study on periodizing the ensemble
Venue: arXiv cs.CE 2021
Year: 2021

We investigate volume-element sampling strategies for the stochastic homogenization of particle-reinforced composites and show, via computational experiments, that an improper treatment of particles intersecting the boundary of the computational cell may affect the accuracy of the computed effective properties. Motivated by recent results on a superior convergence rate of the systematic error for periodized ensembles compared to taking snapshots of ensembles, we conduct computational experiments for microstructures with circular, spherical and cylindrical inclusions and monitor the systematic errors in the effective thermal conductivity for snapshots of ensembles compared to working with microstructures sampled from periodized ensembles. We observe that the standard deviation of the apparent properties computed on microstructures sampled from the periodized ensembles decays at the scaling expected from the central limit theorem. In contrast, the standard deviation for the snapshot ensembles shows an inferior decay rate at high filler content. The latter effect is caused by additional long-range correlations that necessarily appear in particle-reinforced composites at high, industrially relevant, volume fractions. Periodized ensembles, however, appear to be less affected by these correlations. Our findings provide guidelines for working with digital volume images of material microstructures and the design of representative volume elements for computational homogenization.
Attack as Defense: Characterizing Adversarial Examples using Robustness
Venue: arXiv cs.CR 2021
Year: 2021

As a new programming paradigm, deep learning has expanded its application to many real-world problems. At the same time, deep learning based software are found to be vulnerable to adversarial attacks. Though various defense mechanisms have been proposed to improve robustness of deep learning software, many of them are ineffective against adaptive attacks. In this work, we propose a novel characterization to distinguish adversarial examples from benign ones based on the observation that adversarial examples are significantly less robust than benign ones. As existing robustness measurement does not scale to large networks, we propose a novel defense framework, named attack as defense (A2D), to detect adversarial examples by effectively evaluating an example's robustness. A2D uses the cost of attacking an input for robustness evaluation and identifies those less robust examples as adversarial since less robust examples are easier to attack. Extensive experiment results on MNIST, CIFAR10 and ImageNet show that A2D is more effective than recent promising approaches. We also evaluate our defence against potential adaptive attacks and show that A2D is effective in defending carefully designed adaptive attacks, e.g., the attack success rate drops to 0% on CIFAR10.
Improving Diversity of Neural Text Generation via Inverse Probability Weighting
Venue: arXiv cs.CL 2021
Year: 2021

The neural text generation suffers from the text degeneration issue such as repetition. Traditional stochastic sampling methods only focus on truncating the unreliable "tail" of the distribution, and do not address the "head" part, which we show might contain tedious or even repetitive candidates with high probability that lead to repetition loops. They also do not consider the issue that human text does not always favor high-probability words. Inspired by these, in this work we propose a heuristic sampling method. We propose to use interquartile range of the predicted distribution to determine the "head" part, then permutate and rescale the "head" with inverse probability. This aims at decreasing the probability for the tedious and possibly repetitive candidates with higher probability, and increasing the probability for the rational but more surprising candidates with lower probability. The proposed algorithm provides a reasonable permutation on the predicted distribution which enhances diversity without compromising rationality of the distribution. We use pre-trained language model to compare our algorithm with traditional methods. Results show that our algorithm can effectively increase the diversity of generated samples while achieving close resemblance to human text.
Understanding User Topic Preferences across Multiple Social Networks
Venue: arXiv cs.SI 2021
Year: 2021

In recent years, social networks have shown diversity in function and applications. People begin to use multiple online social networks simultaneously for different demands. The ability to uncover a user's latent topic and social network preference is critical for community detection, recommendation, and personalized service across social networks. Unfortunately, most current works focus on the single network, necessitating new technology and models to address this issue. This paper proposes a user preference discovery model on multiple social networks. Firstly, the global and local topic concepts are defined, then a latent semantic topic discovery method is used to obtain global and local topic word distributions, along with user topic and social network preferences. After that, the topic distribution characteristics of different social networks are examined, as well as the reasons why users choose one network over another to create a post. Next, a Gibbs sampling algorithm is adopted to obtain the model parameters. In the experiment, we collect data from Twitter, Instagram, and Tumblr websites to build a dataset of multiple social networks. Finally, we compare our research to previous works, and both qualitative and quantitative evaluation results have demonstrated the effectiveness.
Delegation to autonomous agents promotes cooperation in collective-risk dilemmas
Venue: arXiv cs.HC 2021
Year: 2021

Home assistant chat-bots, self-driving cars, drones or automated negotiations are some of the several examples of autonomous (artificial) agents that have pervaded our society. These agents enable the automation of multiple tasks, saving time and (human) effort. However, their presence in social settings raises the need for a better understanding of their effect on social interactions and how they may be used to enhance cooperation towards the public good, instead of hindering it. To this end, we present an experimental study of human delegation to autonomous agents and hybrid human-agent interactions centered on a public goods dilemma shaped by a collective risk. Our aim to understand experimentally whether the presence of autonomous agents has a positive or negative impact on social behaviour, fairness and cooperation in such a dilemma. Our results show that cooperation increases when participants delegate their actions to an artificial agent that plays on their behalf. Yet, this positive effect is reduced when humans interact in hybrid human-agent groups. Finally, we show that humans are biased towards agent behaviour, assuming that they will contribute less to the collective effort.
Learning Novel Objects Continually Through Curiosity
Venue: arXiv cs.RO 2021
Year: 2021

Children learn continually by asking questions about the concepts they are most curious about. With robots becoming an integral part of our society, they must also learn unknown concepts continually by asking humans questions. The paper analyzes a recent state-of-the-art approach for continual learning. The paper further develops a self-supervised technique to find most of the uncertain objects in an environment by utilizing the cluster representation of the previously learned classes. We test our approach on a benchmark dataset for continual learning on robots. Our results show that our curiosity-driven continual learning approach beats random sampling and softmax-based uncertainty sampling in terms of classification accuracy and the total number of classes learned.
An Empirical Study of Ageing in the Cloud
Venue: arXiv cs.NI 2021
Year: 2021

We quantify, over inter-continental paths, the ageing of TCP packets, throughput and delay for different TCP congestion control algorithms containing a mix of loss-based, delay-based and hybrid congestion control algorithms. In comparing these TCP variants to ACP+, an improvement over ACP, we shed better light on the ability of ACP+ to deliver timely updates over fat pipes and long paths. ACP+ estimates the network conditions on the end-to-end path and adapts the rate of status updates to minimize age. It achieves similar average age as the best (age wise) performing TCP algorithm but at end-to-end throughputs that are two orders of magnitude smaller. We also quantify the significant improvements that ACP+ brings to age control over a shared multiaccess channel.