OpenRaiser/Intern-Atlas huggingface dataset:
ReCode: Robustness Evaluation of Code Generation Models
Venue: arXiv cs.LG 2022
Year: 2022

Code generation models have achieved impressive performance. However, they tend to be brittle as slight edits to a prompt could lead to very different generations; these robustness properties, critical for user experience when deployed in real-life applications, are not well understood. Most existing works on robustness in text or code tasks have focused on classification, while robustness in generation tasks is an uncharted area and to date there is no comprehensive benchmark for robustness in code generation. In this paper, we propose ReCode, a comprehensive robustness evaluation benchmark for code generation models. We customize over 30 transformations specifically for code on docstrings, function and variable names, code syntax, and code format. They are carefully designed to be natural in real-life coding practice, preserve the original semantic meaning, and thus provide multifaceted assessments of a model's robustness performance. With human annotators, we verified that over 90% of the perturbed prompts do not alter the semantic meaning of the original prompt. In addition, we define robustness metrics for code generation models considering the worst-case behavior under each type of perturbation, taking advantage of the fact that executing the generated code can serve as objective evaluation. We demonstrate ReCode on SOTA models using HumanEval, MBPP, as well as function completion tasks derived from them. Interesting observations include: better robustness for CodeGen over InCoder and GPT-J; models are most sensitive to syntax perturbations; more challenging robustness evaluation on MBPP over HumanEval.
Identifying and Manipulating the Personality Traits of Language Models
Venue: arXiv cs.AI 2022
Year: 2022

Psychology research has long explored aspects of human personality such as extroversion, agreeableness and emotional stability. Categorizations like the `Big Five' personality traits are commonly used to assess and diagnose personality types. In this work, we explore the question of whether the perceived personality in language models is exhibited consistently in their language generation. For example, is a language model such as GPT2 likely to respond in a consistent way if asked to go out to a party? We also investigate whether such personality traits can be controlled. We show that when provided different types of contexts (such as personality descriptions, or answers to diagnostic questions about personality traits), language models such as BERT and GPT2 can consistently identify and reflect personality markers in those contexts. This behavior illustrates an ability to be manipulated in a highly predictable way, and frames them as tools for identifying personality traits and controlling personas in applications such as dialog systems. We also contribute a crowd-sourced data-set of personality descriptions of human subjects paired with their `Big Five' personality assessment data, and a data-set of personality descriptions collated from Reddit.
Quantifying and Visualizing Vascular Branching Geometry with Micro-CT: Normalization of Intra- and Inter-Specimen Variations
Venue: arXiv cs.CV 2022
Year: 2022

Micro-CT images of the renal arteries of intact rat kidneys, which had their vasculature injected with the contrast agent polymer Microfil, were characterized. Measurement of inter-branch segment properties and the hierarchical structure of the vessel trees were computed by an automated algorithmic approach. The perfusion territories of the different kidneys, as well as the local diameters of the segmented vasculature were mapped onto the representative structures and visually explored. Various parameters were compared in order to outline key geometrical properties, properties which were shown to not have a wide range of inter-specimen variation. It is shown that the fractal scaling in non-symmetric branching reveals itself differently, than in symmetric branching (e.g., in the lung the mean bronchial diameters at each generation are closely related). Also, perfused tissue is shown to have very little inter-specimen variation and therefore could be used in future studies related to characterizing various disease states of tissues and organs based on vascular branching geometry.
Distributed Key Generation with Smart Contracts using zk-SNARKs
Venue: arXiv cs.CR 2022
Year: 2022

Distributed Key Generation (DKG) is an extensively researched topic as it is fundamental to threshold cryptosystems. Emerging technologies such as blockchains benefit massively from applying threshold cryptography in consensus protocols, randomness beacons, and threshold signatures. However, blockchains and smart contracts also enable further improvements of DKG protocols by providing a decentralized computation and communication platform. For that reason, we propose a DKG protocol that uses smart contracts to ensure the correct execution of the protocol, allow dynamic participation, and provide crypto-economic incentives to encourage honest behavior. The DKG protocol uses a dispute and key derivation mechanism based on Zero-Knowledge Succinct Non-interactive Arguments of Knowledge (zk-SNARKs) to reduce the costs of applying smart contracts by moving the computations off-chain, where the smart contract only verifies the correctness of the computation.
A World Full of Privacy and Security (Mis)conceptions? Findings of a Representative Survey in 12 Countries
Venue: arXiv cs.CR 2022
Year: 2022

Misconceptions about digital security and privacy topics in the general public frequently lead to insecure behavior. However, little is known about the prevalence and extent of such misconceptions in a global context. In this work, we present the results of the first large-scale survey of a global population on misconceptions: We conducted an online survey with n = 12, 351 participants in 12 countries on four continents. By investigating influencing factors of misconceptions around eight common security and privacy topics (including E2EE, Wi-Fi, VPN, and malware), we find the country of residence to be the strongest estimate for holding misconceptions. We also identify differences between non-Western and Western countries, demonstrating the need for region-specific research on user security knowledge, perceptions, and behavior. While we did not observe many outright misconceptions, we did identify a lack of understanding and uncertainty about several fundamental privacy and security topics.
Fine-Grained Distillation for Long Document Retrieval
Venue: arXiv cs.IR 2022
Year: 2022

Long document retrieval aims to fetch query-relevant documents from a large-scale collection, where knowledge distillation has become de facto to improve a retriever by mimicking a heterogeneous yet powerful cross-encoder. However, in contrast to passages or sentences, retrieval on long documents suffers from the scope hypothesis that a long document may cover multiple topics. This maximizes their structure heterogeneity and poses a granular-mismatch issue, leading to an inferior distillation efficacy. In this work, we propose a new learning framework, fine-grained distillation (FGD), for long-document retrievers. While preserving the conventional dense retrieval paradigm, it first produces global-consistent representations crossing different fine granularity and then applies multi-granular aligned distillation merely during training. In experiments, we evaluate our framework on two long-document retrieval benchmarks, which show state-of-the-art performance.
The Expertise Level
Venue: arXiv cs.AI 2022
Year: 2022

Computers are quickly gaining on us. Artificial systems are now exceeding the performance of human experts in several domains. However, we do not yet have a deep definition of expertise. This paper examines the nature of expertise and presents an abstract knowledge-level and skill-level description of expertise. A new level lying above the Knowledge Level, called the Expertise Level, is introduced to describe the skills of an expert without having to worry about details of the knowledge required. The Model of Expertise is introduced combining the knowledge-level and expertise-level descriptions. Application of the model to the fields of cognitive architectures and human cognitive augmentation is demonstrated and several famous intelligent systems are analyzed with the model.
Wireless Power Transfer Analysis using Ansys
Venue: arXiv cs.NI 2022
Year: 2022

In this report, methods of wireless power transfer of inside a cage is compared and simulated. Software Ansys Electronics Maxwell was used to simulate magnetic coupling of transmitter(s) and receiver with 10mA winding current in the mouse cage.
Pragmatic isomorphism proofs between Coq representations: application to lambda-term families
Venue: arXiv cs.LO 2022
Year: 2022

There are several ways to formally represent families of data, such as lambda terms, in a type theory such as the dependent type theory of Coq. Mathematical representations are very compact ones and usually rely on the use of dependent types, but they tend to be difficult to handle in practice. On the contrary, implementations based on a larger (and simpler) data structure combined with a restriction property are much easier to deal with. In this work, we study several families related to lambda terms, among which Motzkin trees, seen as lambda term skeletons, closable Motzkin trees, corresponding to closed lambda terms, and a parameterized family of open lambda terms. For each of these families, we define two different representations, show that they are isomorphic and provide tools to switch from one representation to another. All these datatypes and their associated transformations are implemented in the Coq proof assistant. Furthermore we implement random generators for each representation, using the QuickChick plugin.
PoissonMat: Remodeling Matrix Factorization using Poisson Distribution and Solving the Cold Start Problem without Input Data
Venue: arXiv cs.IR 2022
Year: 2022

Matrix Factorization is one of the most successful recommender system techniques over the past decade. However, the classic probabilistic theory framework for matrix factorization is modeled using normal distributions. To find better probabilistic models, algorithms such as RankMat, ZeroMat and DotMat have been invented in recent years. In this paper, we model the user rating behavior in recommender system as a Poisson process, and design an algorithm that relies on no input data to solve the recommendation problem and the cold start issue at the same time. We prove the superiority of our algorithm in comparison with matrix factorization, random placement, Zipf placement, ZeroMat, DotMat, etc.
Controllable Text Generation with Language Constraints
Venue: arXiv cs.CL 2022
Year: 2022

We consider the task of text generation in language models with constraints specified in natural language. To this end, we first create a challenging benchmark Cognac that provides as input to the model a topic with example text, along with a constraint on text to be avoided. Unlike prior work, our benchmark contains knowledge-intensive constraints sourced from databases like Wordnet and Wikidata, which allows for straightforward evaluation while striking a balance between broad attribute-level and narrow lexical-level controls. We find that even state-of-the-art language models like GPT-3 fail often on this task, and propose a solution to leverage a language model's own internal knowledge to guide generation. Our method, called CognacGen, first queries the language model to generate guidance terms for a specified topic or constraint, and uses the guidance to modify the model's token generation probabilities. We propose three forms of guidance (binary verifier, top-k tokens, textual example), and employ prefix-tuning approaches to distill the guidance to tackle diverse natural language constraints. Through extensive empirical evaluations, we demonstrate that CognacGen can successfully generalize to unseen instructions and outperform competitive baselines in generating constraint conforming text.
ECoHeN: A Hypothesis Testing Framework for Extracting Communities from Heterogeneous Networks
Venue: arXiv cs.SI 2022
Year: 2022

Community discovery is the general process of attaining assortative communities from a network: collections of nodes that are densely connected within yet sparsely connected to the rest of the network. While community discovery has been well studied, few such techniques exist for heterogeneous networks, which contain different types of nodes and possibly different connectivity patterns between the node types. In this paper, we introduce a framework called ECoHeN, which \textbf{e}xtracts \textbf{co}mmunities from a \textbf{he}terogeneous \textbf{n}etwork in a statistically meaningful way. Using a heterogeneous configuration model as a reference distribution, ECoHeN identifies communities that are significantly more densely connected than expected given the node types and connectivity of its membership. Specifically, the ECoHeN algorithm extracts communities one at a time through a dynamic set of iterative updating rules, is guaranteed to converge, and imposes no constraints on the type composition of extracted communities. To our knowledge this is the first discovery method that distinguishes and identifies both homogeneous and heterogeneous, possibly overlapping, community structure in a network. We demonstrate the performance of ECoHeN through simulation and in application to a political blogs network to identify collections of blogs which reference one another more than expected considering the ideology of its' members.
Visual Analytics for Early Detection of Retinal Diseases
Venue: arXiv cs.HC 2022
Year: 2022

Advances in optical coherence tomography (OCT) have enabled noninvasive imaging of substructures of the human retina with high spatial resolution. OCT examinations are now a standard procedure in clinics and an integral part of ophthalmic research. The interpretation of the OCT helps ophthalmologists understand the impact of various retinal and systemic diseases on the structure of the retina in a way not previously possible. In the early stages of retinal diseases, however, the identification and analysis of small and localized substructural changes in the retina remains a challenge. We present an overview of novel visual analytics approaches for the interactive exploration of early retinal changes in single and multiple patients, the comparison of the changes with normative data, and automated quantification and measurement of diagnosis-relevant information. We developed these approaches in close collaboration with ophthalmology researchers and industry experts from a leading OCT device manufacturer. As a result, they not only significantly reduced the time and effort required for OCT data analysis, especially in the context of cross-sectional studies, but have also led to several new discoveries published in biomedical journals.
Random linear estimation with rotationally-invariant designs: Asymptotics at high temperature
Venue: arXiv cs.IT 2022
Year: 2022

We study estimation in the linear model $y=Aβ^\star+ε$, in a Bayesian setting where $β^\star$ has an entrywise i.i.d. prior and the design $A$ is rotationally-invariant in law. In the large system limit as dimension and sample size increase proportionally, a set of related conjectures have been postulated for the asymptotic mutual information, Bayes-optimal mean squared error, and TAP mean-field equations that characterize the Bayes posterior mean of $β^\star$. In this work, we prove these conjectures for a general class of signal priors and for arbitrary rotationally-invariant designs $A$, under a "high-temperature" condition that restricts the range of eigenvalues of $A^\top A$. Our proof uses a conditional second-moment method argument, where we condition on the iterates of a version of the Vector AMP algorithm for solving the TAP mean-field equations.
An Evaluation of the State-of-the-Art Software and Hardware Implementations of BIKE
Venue: arXiv cs.CR 2022
Year: 2022

NIST is conducting a process for the standardization of post-quantum cryptosystems, i.e., cryptosystems that are resistant to attacks by both traditional and quantum computers and that can thus substitute the traditional public-key cryptography solutions which are expected to be broken by quantum computers in the next decades. This manuscript provides an overview and a comparison of the existing state-of-the-art implementations of the BIKE QC-MDPC code-based post-quantum KEM, a candidate in NIST's PQC standardization process. We consider both software, hardware, and mixed hardware-software implementations and evaluate their performance and, for hardware ones, their resource utilization.
The SIMO Block Rayleigh Fading Channel Capacity Scaling with Number of Antennas, Bandwidth and Coherence Length
Venue: arXiv cs.IT 2022
Year: 2022

This paper studies the capacity scaling of non-coherent Single-Input Multiple-Output (SIMO) independent and identically distributed (i.i.d.) Rayleigh block fading channels versus bandwidth ($B$), number of receive antennas ($N$) and coherence block length ($L$). In non-coherent channels (without Channel State Information --CSI) capacity scales as $Θ\left(\min(B,\sqrt{NL},N)\right)$. This is achievable using Pilot-Assisted signaling. Energy Modulation signaling rate scales as $Θ\left(\min(B,\sqrt{N})\right)$. If $L$ is fixed while $B$ and $N$ grow, the two expressions grow equally and Energy Modulation achieves the capacity scaling. However, Energy Modulation rate does not scale as the capacity with the variable $L$. The coherent channel capacity with a priori CSI, in turn, scales as $Θ\left(\min(B,N)\right)$. The coherent channel capacity scaling can be fully achieved in non-coherent channels when $L\geqΘ(N)$. In summary, the channel coherence block length plays a pivotal role in modulation selection and the capacity gap between coherent and non-coherent channels. Pilot-Assisted signaling outperforms Energy Modulation's rate scaling versus coherence block length. Only in high mobility scenarios where $L$ is much smaller than the number of antennas ($L\llΘ(\sqrt{N})$), Energy Modulation is effective in non-coherent channels.
AutoMESC: Automatic Framework for Mining and Classifying Ethereum Smart Contract Vulnerabilities and Their Fixes
Venue: arXiv cs.CR 2022
Year: 2022

Due to the risks associated with vulnerabilities in smart contracts, their security has gained significant attention in recent years. However, there is a lack of open datasets on smart contract vulnerabilities and their fixes that allows for data-driven research. Towards this end, we propose an automated method for mining and classifying Ethereum's smart contract vulnerabilities and their corresponding fixes from GitHub and from the Common Vulnerabilities and Exposures (CVE) records in the National Vulnerability Database. We implemented the proposed method in a fully automated framework, which we call AutoMESC. AutoMESC uses seven of the most well-known smart contract security tools to classify and label the collected vulnerabilities based on vulnerability types. Furthermore, it collects metadata that can be used in data-intensive smart contract security research (e.g., vulnerability detection, vulnerability classification, severity prediction, and automated repair). We used AutoMESC to construct a sample dataset and made it publicly available. Currently, the dataset contains 6.7K smart contracts' vulnerability-fix pairs written in Solidity. We assess the quality of the constructed dataset in terms of accuracy, provenance, and relevance, and compare it with existing datasets. AutoMESC is designed to collect data continuously and keep the corresponding dataset up-to-date with newly discovered smart contract vulnerabilities and their fixes from GitHub and CVE records.
There's Plenty of Room Right Here: Biological Systems as Evolved, Overloaded, Multi-scale Machines
Venue: arXiv cs.MA 2022
Year: 2022

The applicability of computational models to the biological world is an active topic of debate. We argue that a useful path forward results from abandoning hard boundaries between categories and adopting an observer-dependent, pragmatic view. Such a view dissolves the contingent dichotomies driven by human cognitive biases (e.g., tendency to oversimplify) and prior technological limitations in favor of a more continuous, gradualist view necessitated by the study of evolution, developmental biology, and intelligent machines. Efforts to re-shape living systems for biomedical or bioengineering purposes require prediction and control of their function at multiple scales. This is challenging for many reasons, one of which is that living systems perform multiple functions in the same place at the same time. We refer to this as "polycomputing" - the ability of the same substrate to simultaneously compute different things. This ability is an important way in which living things are a kind of computer, but not the familiar, linear, deterministic kind; rather, living things are computers in the broad sense of computational materials as reported in the rapidly-growing physical computing literature. We argue that an observer-centered framework for the computations performed by evolved and designed systems will improve the understanding of meso-scale events, as it has already done at quantum and relativistic scales. Here, we review examples of biological and technological polycomputing, and develop the idea that overloading of different functions on the same hardware is an important design principle that helps understand and build both evolved and designed systems. Learning to hack existing polycomputing substrates, as well as evolve and design new ones, will have massive impacts on regenerative medicine, robotics, and computer engineering.
Generation-Augmented Query Expansion For Code Retrieval
Venue: arXiv cs.SE 2022
Year: 2022

Pre-trained language models have achieved promising success in code retrieval tasks, where a natural language documentation query is given to find the most relevant existing code snippet. However, existing models focus only on optimizing the documentation code pairs by embedding them into latent space, without the association of external knowledge. In this paper, we propose a generation-augmented query expansion framework. Inspired by the human retrieval process - sketching an answer before searching, in this work, we utilize the powerful code generation model to benefit the code retrieval task. Specifically, we demonstrate that rather than merely retrieving the target code snippet according to the documentation query, it would be helpful to augment the documentation query with its generation counterpart - generated code snippets from the code generation model. To the best of our knowledge, this is the first attempt that leverages the code generation model to enhance the code retrieval task. We achieve new state-of-the-art results on the CodeSearchNet benchmark and surpass the baselines significantly.