OpenRaiser/Intern-Atlas huggingface dataset:
A graph representation of molecular ensembles for polymer property prediction
Venue: arXiv cs.LG 2022
Year: 2022

Synthetic polymers are versatile and widely used materials. Similar to small organic molecules, a large chemical space of such materials is hypothetically accessible. Computational property prediction and virtual screening can accelerate polymer design by prioritizing candidates expected to have favorable properties. However, in contrast to organic molecules, polymers are often not well-defined single structures but an ensemble of similar molecules, which poses unique challenges to traditional chemical representations and machine learning approaches. Here, we introduce a graph representation of molecular ensembles and an associated graph neural network architecture that is tailored to polymer property prediction. We demonstrate that this approach captures critical features of polymeric materials, like chain architecture, monomer stoichiometry, and degree of polymerization, and achieves superior accuracy to off-the-shelf cheminformatics methodologies. While doing so, we built a dataset of simulated electron affinity and ionization potential values for >40k polymers with varying monomer composition, stoichiometry, and chain architecture, which may be used in the development of other tailored machine learning approaches. The dataset and machine learning models presented in this work pave the path toward new classes of algorithms for polymer informatics and, more broadly, introduce a framework for the modeling of molecular ensembles.
Generic and Trend-aware Curriculum Learning for Relation Extraction in Graph Neural Networks
Venue: arXiv cs.CL 2022
Year: 2022

We present a generic and trend-aware curriculum learning approach for graph neural networks. It extends existing approaches by incorporating sample-level loss trends to better discriminate easier from harder samples and schedule them for training. The model effectively integrates textual and structural information for relation extraction in text graphs. Experimental results show that the model provides robust estimations of sample difficulty and shows sizable improvement over the state-of-the-art approaches across several datasets.
An Integrated Approach for Energy Efficient Handover and Key Distribution Protocol for Secure NC-enabled Small Cells
Venue: arXiv cs.NI 2022
Year: 2022

Future wireless networks must serve dense mobile networks with high data rates, keeping energy requirements to a possible minimum. The small cell-based network architecture and device-to-device (D2D) communication are already being considered part of 5G networks and beyond. In such environments, network coding (NC) can be employed to achieve both higher throughput and energy efficiency. However, NC-enabled systems need to address security challenges specific to NC, such as pollution attacks. All integrity schemes against pollution attacks generally require proper key distribution and management to ensure security in a mobile environment. Additionally, the mobility requirements in small cell environments are more challenging and demanding in terms of signaling overhead. This paper proposes a blockchain-assisted key distribution protocol tailored for MAC-based integrity schemes, which combined with an uplink reference signal (UL RS) handover mechanism, enables energy efficient secure NC. The performance analysis of the protocol during handover scenarios indicates its suitability for ensuring high level of security against pollution attacks in dense small cell environments with multiple adversaries being present. Furthermore, the proposed scheme achieves lower bandwidth and signaling overhead during handover compared to legacy schemes and the signaling cost reduces significantly as the communication progresses, thus enhancing the network's cumulative energy efficiency.
QAPPA: Quantization-Aware Power, Performance, and Area Modeling of DNN Accelerators
Venue: arXiv cs.AR 2022
Year: 2022

As the machine learning and systems community strives to achieve higher energy-efficiency through custom DNN accelerators and model compression techniques, there is a need for a design space exploration framework that incorporates quantization-aware processing elements into the accelerator design space while having accurate and fast power, performance, and area models. In this work, we present QAPPA, a highly parameterized quantization-aware power, performance, and area modeling framework for DNN accelerators. Our framework can facilitate the future research on design space exploration of DNN accelerators for various design choices such as bit precision, processing element type, scratchpad sizes of processing elements, global buffer size, device bandwidth, number of total processing elements in the the design, and DNN workloads. Our results show that different bit precisions and processing element types lead to significant differences in terms of performance per area and energy. Specifically, our proposed lightweight processing elements achieve up to 4.9x more performance per area and energy improvement when compared to INT16 based implementation.
Intuitive and Efficient Human-robot Collaboration via Real-time Approximate Bayesian Inference
Venue: arXiv cs.RO 2022
Year: 2022

The combination of collaborative robots and end-to-end AI, promises flexible automation of human tasks in factories and warehouses. However, such promise seems a few breakthroughs away. In the meantime, humans and cobots will collaborate helping each other. For these collaborations to be effective and safe, robots need to model, predict and exploit human's intents for responsive decision making processes. Approximate Bayesian Computation (ABC) is an analysis-by-synthesis approach to perform probabilistic predictions upon uncertain quantities. ABC includes priors conveniently, leverages sampling algorithms for inference and is flexible to benefit from complex models, e.g. via simulators. However, ABC is known to be computationally too intensive to run at interactive frame rates required for effective human-robot collaboration tasks. In this paper, we formulate human reaching intent prediction as an ABC problem and describe two key performance innovations which allow computations at interactive rates. Our real-world experiments with a collaborative robot set-up, demonstrate the viability of our proposed approach. Experimental evaluations convey the advantages and value of human intent prediction for packing cooperative tasks. Qualitative results show how anticipating human's reaching intent improves human-robot collaboration without compromising safety. Quantitative task fluency metrics confirm the qualitative claims.
Journey of Migrating Millions of Queries on The Cloud
Venue: arXiv cs.DB 2022
Year: 2022

Treasure Data is processing millions of distributed SQL queries every day on the cloud. Upgrading the query engine service at this scale is challenging because we need to migrate all of the production queries of the customers to a new version while preserving the correctness and performance of the data processing pipelines. To ensure the quality of the query engines, we utilize our query logs to build customer-specific benchmarks and replay these queries with real customer data in a secure pre-production environment. To simulate millions of queries, we need effective minimization of test query sets and better reporting of the simulation results to proactively find incompatible changes and performance regression of the new version. This paper describes the overall design of our system and shares various challenges in maintaining the quality of the query engine service on the cloud.
No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RL
Venue: arXiv cs.LG 2022
Year: 2022

The performance of reinforcement learning (RL) agents is sensitive to the choice of hyperparameters. In real-world settings like robotics or industrial control systems, however, testing different hyperparameter configurations directly on the environment can be financially prohibitive, dangerous, or time consuming. We propose a new approach to tune hyperparameters from offline logs of data, to fully specify the hyperparameters for an RL agent that learns online in the real world. The approach is conceptually simple: we first learn a model of the environment from the offline data, which we call a calibration model, and then simulate learning in the calibration model to identify promising hyperparameters. We identify several criteria to make this strategy effective, and develop an approach that satisfies these criteria. We empirically investigate the method in a variety of settings to identify when it is effective and when it fails.
$(O,G)$-granular variable precision fuzzy rough sets based on overlap and grouping functions
Venue: arXiv cs.AI 2022
Year: 2022

Since Bustince et al. introduced the concepts of overlap and grouping functions, these two types of aggregation functions have attracted a lot of interest in both theory and applications. In this paper, the depiction of $(O,G)$-granular variable precision fuzzy rough sets ($(O,G)$-GVPFRSs for short) is first given based on overlap and grouping functions. Meanwhile, to work out the approximation operators efficiently, we give another expression of upper and lower approximation operators by means of fuzzy implications and co-implications. Furthermore, starting from the perspective of construction methods, $(O,G)$-GVPFRSs are represented under diverse fuzzy relations. Finally, some conclusions on the granular variable precision fuzzy rough sets (GVPFRSs for short) are extended to $(O,G)$-GVPFRSs under some additional conditions.
Team formation and team performance: The balance between team freshness and repeat collaboration
Venue: arXiv cs.DL 2022
Year: 2022

Incorporating fresh members in teams is considered a pathway to team creativity. However, whether freshness improves team performance or not remains unclear, as well as the optimal involvement of fresh members for team performance. This study uses a group of authors on the byline of a publication as a proxy for a scientific team. We extend an indicator, i.e., team freshness, to measure the extent to which a scientific team incorporates new members, by calculating the fraction of new collaboration relations established within the team. Based on more than 43 million scientific publications covering more than a half-century of research from Microsoft Academic Graph, this study provides a holistic picture of the current development of team freshness by outlining the temporal evolution of freshness, and its disciplinary distribution. Subsequently, using a multivariable regression approach, we examine the association between team freshness and papers'short-term and long-term citations.The major findings are as follows: (1)team freshness in scientific teams has been increasing in the past half-century; (2)there exists an inverted-U-shaped association between team freshness and papers' citations in all the disciplines and in different periods;(3)the inverted-U-shaped relationship between team freshness and papers' citations is only found in small teams, while, in large teams, team freshness is significantly positively related to papers' citations.
Design of metamaterial-based heat manipulators by isogeometric shape optimization
Venue: arXiv cs.CE 2022
Year: 2022

There has been a growing interest in controlled heat flux manipulation to increase the efficiency of thermal apparatus. Heat manipulators control and manipulate heat flow. A key to the effective performance of these heat manipulators is their thermal design. Such designs can be achieved by a periodic assembly of unit cells (known as metamaterials or meta-structure), whose geometry and material properties can be optimized for a specific objective. In this work, we focus on thermal metamaterial-based heat manipulators such as thermal concentrator (which concentrates the heat flux in a specified region of the domain). The main scope of the current work is to optimize the shape of the heat manipulators using Particle Swarm Optimization (PSO) method. The geometry is defined using NURBS basis functions due to the higher smoothness and continuity and the thermal boundary value problem is solved using Isogeometric Analysis (IGA). Often, nodes as design variables (as in Lagrange finite element method) generate the serrate shapes of boundaries which need to be smoothened later. For the NURBS-based boundary with the control points as design variables, the required smoothness can be predefined through knot vectors and smoothening in the post-processing can be avoided. The optimized shape generated by PSO is compared with the other shape exploited in the literature. The effects of the number of design variables, the thermal conductivity of the materials used, as well as some of the geometry parameters on the optimum shapes are also demonstrated.
Secure Coding via Gaussian Random Fields
Venue: arXiv cs.IT 2022
Year: 2022

Inverse probability problems whose generative models are given by strictly nonlinear Gaussian random fields show the all-or-nothing behavior: There exists a critical rate at which Bayesian inference exhibits a phase transition. Below this rate, the optimal Bayesian estimator recovers the data perfectly, and above it the recovered data becomes uncorrelated. This study uses the replica method from the theory of spin glasses to show that this critical rate is the channel capacity. This interesting finding has a particular application to the problem of secure transmission: A strictly nonlinear Gaussian random field along with random binning can be used to securely encode a confidential message in a wiretap channel. Our large-system characterization demonstrates that this secure coding scheme asymptotically achieves the secrecy capacity of the Gaussian wiretap channel.
Cross-subject Action Unit Detection with Meta Learning and Transformer-based Relation Modeling
Venue: arXiv cs.CV 2022
Year: 2022

Facial Action Unit (AU) detection is a crucial task for emotion analysis from facial movements. The apparent differences of different subjects sometimes mislead changes brought by AUs, resulting in inaccurate results. However, most of the existing AU detection methods based on deep learning didn't consider the identity information of different subjects. The paper proposes a meta-learning-based cross-subject AU detection model to eliminate the identity-caused differences. Besides, a transformer-based relation learning module is introduced to learn the latent relations of multiple AUs. To be specific, our proposed work is composed of two sub-tasks. The first sub-task is meta-learning-based AU local region representation learning, called MARL, which learns discriminative representation of local AU regions that incorporates the shared information of multiple subjects and eliminates identity-caused differences. The second sub-task uses the local region representation of AU of the first sub-task as input, then adds relationship learning based on the transformer encoder architecture to capture AU relationships. The entire training process is cascaded. Ablation study and visualization show that our MARL can eliminate identity-caused differences, thus obtaining a robust and generalized AU discriminative embedding representation. Our results prove that on the two public datasets BP4D and DISFA, our method is superior to the state-of-the-art technology, and the F1 score is improved by 1.3% and 1.4%, respectively.
Regex in a Time of Deep Learning: The Role of an Old Technology in Age Discrimination Detection in Job Advertisements
Venue: arXiv cs.CL 2022
Year: 2022

Deep learning holds great promise for detecting discriminatory language in the public sphere. However, for the detection of illegal age discrimination in job advertisements, regex approaches are still strong performers. In this paper, we investigate job advertisements in the Netherlands. We present a qualitative analysis of the benefits of the 'old' approach based on regexes and investigate how neural embeddings could address its limitations.
Using Internet Measurements to Map the 2022 Ukrainian Refugee Crisis
Venue: arXiv cs.NI 2022
Year: 2022

The conflict in Ukraine, starting in February 2022, began the largest refugee crisis in decades, with millions of Ukrainian refugees crossing the border to neighboring countries and millions of others forced to move within the country. In this paper we present an insight into how Internet measurements can be used to analyze the refugee crisis. Based on preliminary data from the first two months of the war we analyze how measurement data indicates the trends in the flow of refugees from Ukraine to its neighboring countries, and onward to other countries. We believe that these insights can greatly contribute to the ongoing international effort to map the flow of refugees in order to aid and protect them.
Strong Converses using Change of Measure and Asymptotic Markov Chains
Venue: arXiv cs.IT 2022
Year: 2022

The main contribution of this paper is a strong converse result for $K$-hop distributed hypothesis testing against independence with multiple (intermediate) decision centers under a Markov condition. Our result shows that the set of type-II error exponents that can simultaneously be achieved at all the terminals does not depend on the maximum permissible type-I error probabilities. Our strong converse proof is based on a change of measure argument and on the asymptotic proof of specific Markov chains. This proof method can also be used for other converse proofs, and is appealing because it does not require resorting to variational characterizations or blowing-up methods as in previous related proofs.
CTL* model checking for data-aware dynamic systems with arithmetic
Venue: arXiv cs.LO 2022
Year: 2022

The analysis of complex dynamic systems is a core research topic in formal methods and AI, and combined modelling of systems with data has gained increasing importance in applications such as business process management. In addition, process mining techniques are nowadays used to automatically mine process models from event data, often without correctness guarantees. Thus verification techniques for linear and branching time properties are needed to ensure desired behavior. Here we consider data-aware dynamic systems with arithmetic (DDSAs), which constitute a concise but expressive formalism of transition systems with linear arithmetic guards. We present a CTL* model checking procedure for DDSAs that relies on a finite-state abstraction by means of a set of formulas that capture variable configurations. Linear-time verification was shown to be decidable in specific classes of DDSAs where the constraint language or the control flow are suitably confined. We investigate several of these restrictions for the case of CTL*, with both positive and negative results: CTL* verification is proven decidable for monotonicity and integer periodicity constraint systems, but undecidable for feedback free and bounded lookback systems. To demonstrate the feasibility of our approach, we implemented it in the SMT-based prototype ada, showing that many practical business process models can be effectively analyzed.
Formalization of Advanced VOs semantics and VO Refinement
Venue: arXiv cs.LO 2022
Year: 2022

This document lays out the foundations for VO and requirement refinement, abstractions of models, and instantiations. Also, VOs on abstractions and instantiations are considered.
Confidential Machine Learning within Graphcore IPUs
Venue: arXiv cs.CR 2022
Year: 2022

We present IPU Trusted Extensions (ITX), a set of experimental hardware extensions that enable trusted execution environments in Graphcore's AI accelerators. ITX enables the execution of AI workloads with strong confidentiality and integrity guarantees at low performance overheads. ITX isolates workloads from untrusted hosts, and ensures their data and models remain encrypted at all times except within the IPU. ITX includes a hardware root-of-trust that provides attestation capabilities and orchestrates trusted execution, and on-chip programmable cryptographic engines for authenticated encryption of code and data at PCIe bandwidth. We also present software for ITX in the form of compiler and runtime extensions that support multi-party training without requiring a CPU-based TEE. Experimental support for ITX is included in Graphcore's GC200 IPU taped out at TSMC's 7nm technology node. Its evaluation on a development board using standard DNN training workloads suggests that ITX adds less than 5% performance overhead, and delivers up to 17x better performance compared to CPU-based confidential computing systems relying on AMD SEV-SNP.
Distributional Robustness: From Pricing to Auctions
Venue: arXiv cs.GT 2022
Year: 2022

Robust mechanism design is a rising alternative to Bayesian mechanism design, which yields designs that do not rely on assumptions like full distributional knowledge. We apply this approach to mechanisms for selling a single item, assuming that only the mean of the value distribution and an upper bound on the bidder values are known. We seek the mechanism that maximizes revenue over the worst-case distribution compatible with the known parameters. Such a mechanism arises as an equilibrium of a zero-sum game between the seller and an adversary who chooses the distribution, and so can be referred to as the max-min mechanism. Carrasco et al. [2018] derive the max-min pricing when the seller faces a single bidder for the item. We go from max-min pricing to max-min auctions by studying the canonical setting of two i.i.d. bidders, and show the max-min mechanism is the second-price auction with a randomized reserve. We derive a closed-form solution for the distribution over reserve prices, as well as the worst-case value distribution, for which there is simple economic intuition. In fact we derive a closed-form solution for the reserve price distribution for any number of bidders. Our technique for solving the zero-sum game is quite different than that of Carrasco et al.- it involves analyzing a discretized version of the setting, then refining the discretization grid and deriving a closed-form solution for the non-discretized, original setting. Our results establish a difference between the case of two bidders and that of $n \ge 3$ bidders.
Minimising Biasing Word Errors for Contextual ASR with the Tree-Constrained Pointer Generator
Venue: arXiv cs.CL 2022
Year: 2022

Contextual knowledge is essential for reducing speech recognition errors on high-valued long-tail words. This paper proposes a novel tree-constrained pointer generator (TCPGen) component that enables end-to-end ASR models to bias towards a list of long-tail words obtained using external contextual information. With only a small overhead in memory use and computation cost, TCPGen can structure thousands of biasing words efficiently into a symbolic prefix-tree and creates a neural shortcut between the tree and the final ASR output to facilitate the recognition of the biasing words. To enhance TCPGen, we further propose a novel minimum biasing word error (MBWE) loss that directly optimises biasing word errors during training, along with a biasing-word-driven language model discounting (BLMD) method during the test. All contextual ASR systems were evaluated on the public Librispeech audiobook corpus and the data from the dialogue state tracking challenges (DSTC) with the biasing lists extracted from the dialogue-system ontology. Consistent word error rate (WER) reductions were achieved with TCPGen, which were particularly significant on the biasing words with around 40\% relative reductions in the recognition error rates. MBWE and BLMD further improved the effectiveness of TCPGen and achieved more significant WER reductions on the biasing words. TCPGen also achieved zero-shot learning of words not in the audio training set with large WER reductions on the out-of-vocabulary words in the biasing list.