OpenRaiser/Intern-Atlas huggingface dataset:
Statistics in everyone's backyard: an impact study via citation network analysis
Venue: arXiv cs.DL 2021
Year: 2021

The increasing availability of curated citation data provides a wealth of resources for analyzing and understanding the intellectual influence of scientific publications. In the field of statistics, current studies of citation data have mostly focused on the interactions between statistical journals and papers, limiting the measure of influence to mainly within statistics itself. In this paper, we take the first step towards understanding the impact statistics has made on other scientific fields in the era of Big Data. By collecting comprehensive bibliometric data from the Web of Science database for selected statistical journals, we investigate the citation trends and compositions of citing fields over time to show that their diversity has been increasing. Furthermore, we use the local clustering technique involving personalized PageRank with conductance for size selection to find the most relevant statistical research area for a given external topic of interest. We provide theoretical guarantees for the procedure and, through a number of case studies, show the results from our citation data align well with our knowledge and intuition about these external topics. Overall, we have found that the statistical theory and methods recently invented by the statistics community have made increasing impact on other scientific fields.
Modelling Behaviour Change using Cognitive Agent Simulations
Venue: arXiv cs.CY 2021
Year: 2021

In health psychology, Behaviour Change Theories(BCTs) play an important role in modelling human goal achievement in adverse environments. Some of these theories use concepts that are also used in computational modelling of cognition and affect in AI. Examples include dual-process architecture and models of motivation. It is therefore important to ask whether some BCTs can be computationally implemented as cognitive agents in a way that builds on existing AI research in cognitive architecture. This paper presents work-in-progress research to apply selected behaviour change theories to simulated agents, so that an agent is acting according to the theory while attempting to complete a task in a challenging scenario. Two behaviour change theories are selected as examples (CEOS and PRIME). The research is focusing on complex agent architectures required for self-determined goal achievement in adverse circumstances where the action is difficult to maintain (e.g. healthy eating at office parties). Such simulations are useful because they can provide new insights into human behaviour change and improve conceptual precision. In addition, they can act as a rapid-prototyping environment for technology development. High-level descriptive simulations also provide an opportunity for transparency and participatory design, which is important for user ownership of the behaviour change process.
Learning UI Navigation through Demonstrations composed of Macro Actions
Venue: arXiv cs.AI 2021
Year: 2021

We have developed a framework to reliably build agents capable of UI navigation. The state space is simplified from raw-pixels to a set of UI elements extracted from screen understanding, such as OCR and icon detection. The action space is restricted to the UI elements plus a few global actions. Actions can be customized for tasks and each action is a sequence of basic operations conditioned on status checks. With such a design, we are able to train DQfD and BC agents with a small number of demonstration episodes. We propose demo augmentation that significantly reduces the required number of human demonstrations. We made a customization of DQfD to allow demos collected on screenshots to facilitate the demo coverage of rare cases. Demos are only collected for the failed cases during the evaluation of the previous version of the agent. With 10s of iterations looping over evaluation, demo collection, and training, the agent reaches a 98.7\% success rate on the search task in an environment of 80+ apps and websites where initial states and viewing parameters are randomized.
Tackling the Imbalance for GNNs
Venue: arXiv cs.LG 2021
Year: 2021

Different from deep neural networks for non-graph data classification, graph neural networks (GNNs) leverage the information exchange between nodes (or samples) when representing nodes. The category distribution shows an imbalance or even a highly-skewed trend on nearly all existing benchmark GNN data sets. The imbalanced distribution will cause misclassification of nodes in the minority classes, and even cause the classification performance on the entire data set to decrease. This study explores the effects of the imbalance problem on the performances of GNNs and proposes new methodologies to solve it. First, a node-level index, namely, the label difference index ($LDI$), is defined to quantitatively analyze the relationship between imbalance and misclassification. The less samples in a class, the higher the value of its average $LDI$; the higher the $LDI$ of a sample, the more likely the sample will be misclassified. We define a new loss and propose four new methods based on $LDI$. Experimental results indicate that the classification accuracies of the three among our proposed four new methods are better in both transductive and inductive settings. The $LDI$ can be applied to other GNNs.
Novel Secret-Key-Assisted Schemes for Secure MISOME-OFDM Systems
Venue: arXiv cs.IT 2021
Year: 2021

We propose a new secure transmission scheme for uplink multiple-input single-output (MISO) orthogonal-frequency multiplexing (OFDM) systems in the presence of multiple eavesdroppers. Our proposed scheme utilizes the sub-channels orthogonality of OFDM systems to simultaneously transmit data and secret key symbols. The base station, Bob, shares secret key symbols with the legitimate user, Alice, using wiretap coding over a portion of the sub-channels. Concurrently, Alice uses the accumulated secret keys in her secret-key queue to encrypt data symbols using a one time pad (OTP) cipher and transmits them to Bob over the remaining sub-channels. if Alice did not accumulate sufficient keys in her secret-key queue, she employs wiretap coding to secure her data transmissions. We propose fixed and dynamic sub-channel allocation schemes to divide the sub-channels between data and secret keys. We derive the secrecy outage probability (SOP) and the secure throughput for the proposed scheme. We quantify the system's security under practical non-Gaussian transmissions where discrete signal constellation points are transmitted by the legitimate source nodes. Numerical results validate our theoretical findings and quantify the impact of different system design parameters.
Adapting Membership Inference Attacks to GNN for Graph Classification: Approaches and Implications
Venue: arXiv cs.LG 2021
Year: 2021

Graph Neural Networks (GNNs) are widely adopted to analyse non-Euclidean data, such as chemical networks, brain networks, and social networks, modelling complex relationships and interdependency between objects. Recently, Membership Inference Attack (MIA) against GNNs raises severe privacy concerns, where training data can be leaked from trained GNN models. However, prior studies focus on inferring the membership of only the components in a graph, e.g., an individual node or edge. How to infer the membership of an entire graph record is yet to be explored. In this paper, we take the first step in MIA against GNNs for graph-level classification. Our objective is to infer whether a graph sample has been used for training a GNN model. We present and implement two types of attacks, i.e., training-based attacks and threshold-based attacks from different adversarial capabilities. We perform comprehensive experiments to evaluate our attacks in seven real-world datasets using five representative GNN models. Both our attacks are shown effective and can achieve high performance, i.e., reaching over 0.7 attack F1 scores in most cases. Furthermore, we analyse the implications behind the MIA against GNNs. Our findings confirm that GNNs can be even more vulnerable to MIA than the models with non-graph structures. And unlike the node-level classifier, MIAs on graph-level classification tasks are more co-related with the overfitting level of GNNs rather than the statistic property of their training graphs.
S-Cyc: A Learning Rate Schedule for Iterative Pruning of ReLU-based Networks
Venue: arXiv cs.LG 2021
Year: 2021

We explore a new perspective on adapting the learning rate (LR) schedule to improve the performance of the ReLU-based network as it is iteratively pruned. Our work and contribution consist of four parts: (i) We find that, as the ReLU-based network is iteratively pruned, the distribution of weight gradients tends to become narrower. This leads to the finding that as the network becomes more sparse, a larger value of LR should be used to train the pruned network. (ii) Motivated by this finding, we propose a novel LR schedule, called S-Cyclical (S-Cyc) which adapts the conventional cyclical LR schedule by gradually increasing the LR upper bound (max_lr) in an S-shape as the network is iteratively pruned.We highlight that S-Cyc is a method agnostic LR schedule that applies to many iterative pruning methods. (iii) We evaluate the performance of the proposed S-Cyc and compare it to four LR schedule benchmarks. Our experimental results on three state-of-the-art networks (e.g., VGG-19, ResNet-20, ResNet-50) and two popular datasets (e.g., CIFAR-10, ImageNet-200) demonstrate that S-Cyc consistently outperforms the best performing benchmark with an improvement of 2.1% - 3.4%, without substantial increase in complexity. (iv) We evaluate S-Cyc against an oracle and show that S-Cyc achieves comparable performance to the oracle, which carefully tunes max_lr via grid search.
A Framework of Mahalanobis-Distance Metric with Supervised Learning for Clustering Multipath Components in MIMO Channel Analysis
Venue: arXiv cs.IT 2021
Year: 2021

As multipath components (MPCs) are experimentally observed to appear in clusters, cluster-based channel models have been focused in the wireless channel study. However, most of the MPC clustering algorithms for MIMO channels with delay and angle information of MPCs are based on the distance metric that quantifies the similarity of two MPCs and determines the preferred cluster shape, greatly impacting MPC clustering quality. In this paper, a general framework of Mahalanobis-distance metric is proposed for MPC clustering in MIMO channel analysis, without user-specified parameters. Remarkably, the popular multipath component distance (MCD) is proved to be a special case of the proposed distance metric framework. Furthermore, two machine learning algorithms, namely, weak-supervised Mahalanobis metric for clustering and supervised large margin nearest neighbor, are introduced to learn the distance metric. To evaluate the effectiveness, a modified channel model is proposed based on the 3GPP spatial channel model to generate clustered MPCs with delay and angular information, since the original 3GPP spatial channel model (SCM) is incapable to evaluate clustering quality. Experiment results show that the proposed distance metric can significantly improve the clustering quality of existing clustering algorithms, while the learning phase requires considerably limited efforts of labeling MPCs.
A Novel Watermarking Approach for Protecting Image Integrity based on a Hybrid Security Technique
Venue: arXiv cs.CR 2021
Year: 2021

Digital Photo images are everywhere around us in journals, on walls, and over the Internet. However we have to be conscious that seeing does not always imply reality. Photo images become a rich subject of manipulations due to the advanced digital cameras as well as photo editing software. Accordingly, image forgery is becoming much easier using the existing tools in terms of time and accuracy, and thus the forensics of detecting an image forgery case is becoming difficult and needs more and more time and techniques to prove the image originality especially as crime evidences and court related cases. In this paper, a framework with associated algorithms and methodologies is proposed to ensure the authenticity of the image and the integrity of the content in addition to protecting the photo image against forgery suspects. The framework depends on developing new generation of certified digital cameras that could produce authenticated and forgery-proof photos. The proposed methodology generates an irreversible hash integrity code from the image content based on color matrix calculations and steganography algorithms. The simulation results proved the capability of the proposed technique to detect image forgery cases in more than 16 scenarios of manipulation.
Exploring Deep Neural Networks on Edge TPU
Venue: arXiv cs.LG 2021
Year: 2021

This paper explores the performance of Google's Edge TPU on feed forward neural networks. We consider Edge TPU as a hardware platform and explore different architectures of deep neural network classifiers, which traditionally has been a challenge to run on resource constrained edge devices. Based on the use of a joint-time-frequency data representation, also known as spectrogram, we explore the trade-off between classification performance and the energy consumed for inference. The energy efficiency of Edge TPU is compared with that of widely-used embedded CPU ARM Cortex-A53. Our results quantify the impact of neural network architectural specifications on the Edge TPU's performance, guiding decisions on the TPU's optimal operating point, where it can provide high classification accuracy with minimal energy consumption. Also, our evaluations highlight the crossover in performance between the Edge TPU and Cortex-A53, depending on the neural network specifications. Based on our analysis, we provide a decision chart to guide decisions on platform selection based on the model parameters and context.
Towards More Accountable Search Engines: Online Evaluation of Representation Bias
Venue: arXiv cs.IR 2021
Year: 2021

Information availability affects people's behavior and perception of the world. Notably, people rely on search engines to satisfy their need for information. Search engines deliver results relevant to user requests usually without being or making themselves accountable for the information they deliver, which may harm people's lives and, in turn, society. This potential risk urges the development of evaluation mechanisms of bias in order to empower the user in judging the results of search engines. In this paper, we give a possible solution to measuring representation bias with respect to societal features for search engines and apply it to evaluating the gender representation bias for Google's Knowledge Graph Carousel for listing occupations.
HIDE & SEEK: Privacy-Preserving Rebalancing on Payment Channel Networks
Venue: arXiv cs.CR 2021
Year: 2021

Payment channels effectively move the transaction load off-chain thereby successfully addressing the inherent scalability problem most cryptocurrencies face. A major drawback of payment channels is the need to ``top up'' funds on-chain when a channel is depleted. Rebalancing was proposed to alleviate this issue, where parties with depleting channels move their funds along a cycle to replenish their channels off-chain. Protocols for rebalancing so far either introduce local solutions or compromise privacy. In this work, we present an opt-in rebalancing protocol that is both private and globally optimal, meaning our protocol maximizes the total amount of rebalanced funds. We study rebalancing from the framework of linear programming. To obtain full privacy guarantees, we leverage multi-party computation in solving the linear program, which is executed by selected participants to maintain efficiency. Finally, we efficiently decompose the rebalancing solution into incentive-compatible cycles which conserve user balances when executed atomically. Keywords: Payment Channel Networks, Privacy and Rebalancing.
Deep forecasting of translational impact in medical research
Venue: arXiv cs.DL 2021
Year: 2021

The value of biomedical research--a $1.7 trillion annual investment--is ultimately determined by its downstream, real-world impact. Current objective predictors of impact rest on proxy, reductive metrics of dissemination, such as paper citation rates, whose relation to real-world translation remains unquantified. Here we sought to determine the comparative predictability of future real-world translation--as indexed by inclusion in patents, guidelines or policy documents--from complex models of the abstract-level content of biomedical publications versus citations and publication meta-data alone. We develop a suite of representational and discriminative mathematical models of multi-scale publication data, quantifying predictive performance out-of-sample, ahead-of-time, across major biomedical domains, using the entire corpus of biomedical research captured by Microsoft Academic Graph from 1990 to 2019, encompassing 43.3 million papers across all domains. We show that citations are only moderately predictive of translational impact as judged by inclusion in patents, guidelines, or policy documents. By contrast, high-dimensional models of publication titles, abstracts and metadata exhibit high fidelity (AUROC > 0.9), generalise across time and thematic domain, and transfer to the task of recognising papers of Nobel Laureates. The translational impact of a paper indexed by inclusion in patents, guidelines, or policy documents can be predicted--out-of-sample and ahead-of-time--with substantially higher fidelity from complex models of its abstract-level content than from models of publication meta-data or citation metrics. We argue that content-based models of impact are superior in performance to conventional, citation-based measures, and sustain a stronger evidence-based claim to the objective measurement of translational potential.
Real Time Cluster Path Tracing
Venue: arXiv cs.GR 2021
Year: 2021

Photorealistic rendering effects are common in films, but most real time graphics today still rely on scan-line based multi-pass rendering to deliver rich visual experiences. While there have been prior works in distributed path tracing for static scene and objects under rigid motion, real time path tracing of deforming characters has to support per-frame dynamic BVH changes. We present the architecture and implementation of the first real-time production quality cluster path tracing renderer that supports film quality animation and deformation. We build our cluster path tracing system using the open source Blender and its GPU accelerated production quality renderer Cycles. Our system's rendering performance and quality scales linearly with the number of RTX cluster nodes used. It is able to generate and deliver path traced images with global illumination effects to remote light-weight client systems at 15-30 frames per second for a variety of Blender scenes including animated digital human characters with skin deformation and virtual objects.
Empirical Policy Optimization for $n$-Player Markov Games
Venue: arXiv cs.GT 2021
Year: 2021

In single-agent Markov decision processes, an agent can optimize its policy based on the interaction with environment. In multi-player Markov games (MGs), however, the interaction is non-stationary due to the behaviors of other players, so the agent has no fixed optimization objective. In this paper, we treat the evolution of player policies as a dynamical process and propose a novel learning scheme for Nash equilibrium. The core is to evolve one's policy according to not just its current in-game performance, but an aggregation of its performance over history. We show that for a variety of MGs, players in our learning scheme will provably converge to a point that is an approximation to Nash equilibrium. Combined with neural networks, we develop the \emph{empirical policy optimization} algorithm, that is implemented in a reinforcement-learning framework and runs in a distributed way, with each player optimizing its policy based on own observations. We use two numerical examples to validate the convergence property on small-scale MGs with $n\ge 2$ players, and a pong example to show the potential of our algorithm on large games.
StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis
Venue: arXiv cs.CV 2021
Year: 2021

We propose StyleNeRF, a 3D-aware generative model for photo-realistic high-resolution image synthesis with high multi-view consistency, which can be trained on unstructured 2D images. Existing approaches either cannot synthesize high-resolution images with fine details or yield noticeable 3D-inconsistent artifacts. In addition, many of them lack control over style attributes and explicit 3D camera poses. StyleNeRF integrates the neural radiance field (NeRF) into a style-based generator to tackle the aforementioned challenges, i.e., improving rendering efficiency and 3D consistency for high-resolution image generation. We perform volume rendering only to produce a low-resolution feature map and progressively apply upsampling in 2D to address the first issue. To mitigate the inconsistencies caused by 2D upsampling, we propose multiple designs, including a better upsampler and a new regularization loss. With these designs, StyleNeRF can synthesize high-resolution images at interactive rates while preserving 3D consistency at high quality. StyleNeRF also enables control of camera poses and different levels of styles, which can generalize to unseen views. It also supports challenging tasks, including zoom-in and-out, style mixing, inversion, and semantic editing.
Natural Image Reconstruction from fMRI using Deep Learning: A Survey
Venue: arXiv cs.CV 2021
Year: 2021

With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics and present a fair performance evaluation across standardized evaluation metrics. Finally, we discuss the strengths and limitations of existing studies and present potential future directions.
Arjun: An Efficient Independent Support Computation Technique and its Applications to Counting and Sampling
Venue: arXiv cs.AI 2021
Year: 2021

Given a Boolean formula $φ$ over the set of variables $X$ and a projection set $\mathcal{P} \subseteq X$, a subset of variables $\mathcal{I}$ is independent support of $\mathcal{P}$ if two solutions agree on $\mathcal{I}$, then they also agree on $\mathcal{P}$. The notion of independent support is related to the classical notion of definability dating back to 1901, and have been studied over the decades. Recently, the computational problem of determining independent support for a given formula has attained importance owing to the crucial importance of independent support for hashing-based counting and sampling techniques. In this paper, we design an efficient and scalable independent support computation technique that can handle formulas arising from real-world benchmarks. Our algorithmic framework, called Arjun, employs implicit and explicit definability notions, and is based on a tight integration of gate-identification techniques and assumption-based framework. We demonstrate that augmenting the state of the art model counter ApproxMC4 and sampler UniGen3 with Arjun leads to significant performance improvements. In particular, ApproxMC4 augmented with Arjun counts 387 more benchmarks out of 1896 while UniGen3 augmented with Arjun samples 319 more benchmarks within the same time limit.