pwc-archive/papers-with-abstracts huggingface dataset:
SpaceTx: A Roadmap for Benchmarking Spatial Transcriptomics Exploration of the Brain
Authors: Brian Long, Jeremy Miller, The SpaceTx Consortium
Date: 2023-01-20 00:00:00
Tasks: Benchmarking, Cell Segmentation

Mapping spatial distributions of transcriptomic cell types is essential to understanding the brain, with its exceptional cellular heterogeneity and the functional significance of its spatial organization. Spatial transcriptomics techniques are hoped to accomplish these measurements, but each method uses different experimental and computational protocols, with different trade-offs and optimizations. In 2017, the SpaceTx Consortium was formed to compare these methods and determine their suitability for large-scale spatial transcriptomic atlases. SpaceTx work included progress in tissue processing, taxonomy development, gene selection, image processing and data standardization, cell segmentation, cell type assignments, and visualization. Although the landscape of experimental methods has changed dramatically since the beginning of SpaceTx, the need for quantitative and detailed benchmarking of spatial transcriptomics methods in the brain is still unmet. Here, we summarize the work of SpaceTx and highlight outstanding challenges as spatial transcriptomics grows into a mature field. We also discuss how our progress provides a roadmap for benchmarking spatial transcriptomics methods in the future. Data and analyses from this consortium, along with code and methods are publicly available at https://spacetx.github.io/.
AI of Brain and Cognitive Sciences: From the Perspective of First Principles
Authors: Luyao Chen, Zhiqiang Chen, Longsheng Jiang, Xiang Liu, Linlu Xu, Bo Zhang, Xiaolong Zou, Jinying Gao, Yu Zhu, Xizi Gong, Shan Yu, Sen Song, Liangyi Chen, Fang Fang, Si Wu, Jia Liu
Date: 2023-01-20 00:00:00
Tasks: Few-Shot Learning, image-classification, Image Classification

Nowadays, we have witnessed the great success of AI in various applications, including image classification, game playing, protein structure analysis, language translation, and content generation. Despite these powerful applications, there are still many tasks in our daily life that are rather simple to humans but pose great challenges to AI. These include image and language understanding, few-shot learning, abstract concepts, and low-energy cost computing. Thus, learning from the brain is still a promising way that can shed light on the development of next-generation AI. The brain is arguably the only known intelligent machine in the universe, which is the product of evolution for animals surviving in the natural environment. At the behavior level, psychology and cognitive sciences have demonstrated that human and animal brains can execute very intelligent high-level cognitive functions. At the structure level, cognitive and computational neurosciences have unveiled that the brain has extremely complicated but elegant network forms to support its functions. Over years, people are gathering knowledge about the structure and functions of the brain, and this process is accelerating recently along with the initiation of giant brain projects worldwide. Here, we argue that the general principles of brain functions are the most valuable things to inspire the development of AI. These general principles are the standard rules of the brain extracting, representing, manipulating, and retrieving information, and here we call them the first principles of the brain. This paper collects six such first principles. They are attractor network, criticality, random network, sparse coding, relational memory, and perceptual learning. On each topic, we review its biological background, fundamental property, potential application to AI, and future development.
Geometric approach for non pharmaceutical interventions in epidemiology
Authors: Laurent Evain, Jean-Jacques Loeb
Date: 2023-01-20 00:00:00
Tasks: Epidemiology

Various non pharmaceutical interventions have been settled to minimise the burden of the COVID-19 outbreak. We build a framework to analyse the dynamics of non pharmaceutical interventions, to distinguish between mitigations measures leading to objective scientific improvements and mitigations based on both political and scientific considerations. We analyse two possible strategies within this framework. Namely, we consider mitigations driven by the limited resources of the health system and mitigations where a constant set of measures is applied at different moments. We describe the optimal interventions for these scenarios. Our approach is mathematical and involves sir differential systems, it is qualitative and geometrical rather than computational. Along with the analysis of these scenarios, we collect several results that may be useful on their own, in particular on the ground when the variables are not known in real time.
Domain-adapted Learning and Interpretability: DRL for Gas Trading
Authors: Yuanrong Wang, Yinsen Miao, Alexander CY Wong, Nikita P Granger, Christian Michler
Date: 2023-01-19 00:00:00
Tasks: Deep Reinforcement Learning, Ensemble Learning, reinforcement-learning, Reinforcement Learning (RL)

Deep Reinforcement Learning (Deep RL) has been explored for a number of applications in finance and stock trading. In this paper, we present a practical implementation of Deep RL for trading natural gas futures contracts. The Sharpe Ratio obtained exceeds benchmarks given by trend following and mean reversion strategies as well as results reported in literature. Moreover, we propose a simple but effective ensemble learning scheme for trading, which significantly improves performance through enhanced model stability and robustness as well as lower turnover and hence lower transaction cost. We discuss the resulting Deep RL strategy in terms of model explainability, trading frequency and risk measures.
Robot Skill Learning Via Classical Robotics-Based Generated Datasets: Advantages, Disadvantages, and Future Improvement
Authors: Batu Kaan Oezen
Date: 2023-01-20 00:00:00
Tasks: Domain Adaptation

Why do we not profit from our long-existing classical robotics knowledge and look for some alternative way for data collection? The situation ignoring all existing methods might be such a waste. This article argues that a dataset created using a classical robotics algorithm is a crucial part of future development. This developed classic algorithm has a perfect domain adaptation and generalization property, and most importantly, collecting datasets based on them is quite easy. It is well known that current robot skill-learning approaches perform exceptionally badly in the unseen domain, and their performance against adversarial attacks is quite limited as long as they do not have a very exclusive big dataset. Our experiment is the initial steps of using a dataset created by classical robotics codes. Our experiment investigated possible trajectory collection based on classical robotics. It addressed some advantages and disadvantages and pointed out other future development ideas.
Propagation of Linear Uncertainties through Multiline Thru-Reflect-Line Calibration
Authors: Ziad Hatab, Michael Ernst Gadringer, Wolfgang Bösch
Date: 2023-01-22 00:00:00

This study proposes a linear approach for propagating uncertainties in the multiline thru-reflect-line (TRL) calibration method for vector network analyzers. The multiline TRL formulation we are proposing applies the law of uncertainty propagation as outlined in the ISO Guide to the Expression of Uncertainty in Measurement (GUM) to both measurement and model uncertainties. In addition, we conducted a Monte Carlo analysis using a combination of measured and synthetic data to model various uncertainties, such as additive noise, reflect asymmetry, line mismatch, and line length offset. The results of our linear uncertainty formulation demonstrate agreement with the Monte Carlo method and provide a more efficient means of assessing the uncertainty budget of the multiline TRL calibration.
An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models
Authors: Saghar Hosseini, Hamid Palangi, Ahmed Hassan Awadallah
Date: 2023-01-22 00:00:00
Tasks: Language Modeling, Language Modelling

Large-scale Pre-Trained Language Models (PTLMs) capture knowledge from massive human-written data which contains latent societal biases and toxic contents. In this paper, we leverage the primary task of PTLMs, i.e., language modeling, and propose a new metric to quantify manifested implicit representational harms in PTLMs towards 13 marginalized demographics. Using this metric, we conducted an empirical analysis of 24 widely used PTLMs. Our analysis provides insights into the correlation between the proposed metric in this work and other related metrics for representational harm. We observe that our metric correlates with most of the gender-specific metrics in the literature. Through extensive experiments, we explore the connections between PTLMs architectures and representational harms across two dimensions: depth and width of the networks. We found that prioritizing depth over width, mitigates representational harms in some PTLMs. Our code and data can be found at https://github.com/microsoft/SafeNLP.
Optimising Event-Driven Spiking Neural Network with Regularisation and Cutoff
Authors: Dengyu Wu, Gaojie Jin, Han Yu, Xinping Yi, Xiaowei Huang
Date: 2023-01-23 00:00:00
Tasks: Computational Efficiency

Spiking neural network (SNN), as the next generation of artificial neural network (ANN), offer a closer mimicry of natural neural networks and hold promise for significant improvements in computational efficiency. However, the current SNN is trained to infer over a fixed duration, overlooking the potential of dynamic inference in SNN. In this paper, we strengthen the marriage between SNN and event-driven processing with a proposal to consider a cutoff in SNN, which can terminate SNN anytime during inference to achieve efficient inference. Two novel optimisation techniques are presented to achieve inference efficient SNN: a Top-K cutoff and a regularisation.The proposed regularisation influences the training process, optimising SNN for the cutoff, while the Top-K cutoff technique optimises the inference phase. We conduct an extensive set of experiments on multiple benchmark frame-based datasets, such asCIFAR10/100, Tiny-ImageNet, and event-based datasets, including CIFAR10-DVS, N-Caltech101 and DVS128 Gesture. The experimental results demonstrate the effectiveness of our techniques in both ANN-to-SNN conversion and direct training, enabling SNNs to require 1.76 to 2.76x fewer timesteps for CIFAR-10, while achieving 1.64 to 1.95x fewer timesteps across all event-based datasets, with near-zero accuracy loss. These findings affirms the compatibility and potential benefits of our techniques in enhancing accuracy and reducing inference latency when integrated with existing methods. Code available: https://github.com/Dengyu-Wu/SNNCutoff
SPEC5G: A Dataset for 5G Cellular Network Protocol Analysis
Authors: Imtiaz Karim, Kazi Samin Mubasshir, Mirza Masfiqur Rahman, Elisa Bertino
Date: 2023-01-22 00:00:00
Tasks: text-classification, Text Classification

5G is the 5th generation cellular network protocol. It is the state-of-the-art global wireless standard that enables an advanced kind of network designed to connect virtually everyone and everything with increased speed and reduced latency. Therefore, its development, analysis, and security are critical. However, all approaches to the 5G protocol development and security analysis, e.g., property extraction, protocol summarization, and semantic analysis of the protocol specifications and implementations are completely manual. To reduce such manual effort, in this paper, we curate SPEC5G the first-ever public 5G dataset for NLP research. The dataset contains 3,547,586 sentences with 134M words, from 13094 cellular network specifications and 13 online websites. By leveraging large-scale pre-trained language models that have achieved state-of-the-art results on NLP tasks, we use this dataset for security-related text classification and summarization. Security-related text classification can be used to extract relevant security-related properties for protocol testing. On the other hand, summarization can help developers and practitioners understand the high level of the protocol, which is itself a daunting task. Our results show the value of our 5G-centric dataset in 5G protocol analysis automation. We believe that SPEC5G will enable a new research direction into automatic analyses for the 5G cellular network protocol and numerous related downstream tasks. Our data and code are publicly available.
FInC Flow: Fast and Invertible $k \times k$ Convolutions for Normalizing Flows
Authors: Aditya Kallappa, Sandeep Nagar, Girish Varma
Date: 2023-01-23 00:00:00
Tasks: Image Generation

Invertible convolutions have been an essential element for building expressive normalizing flow-based generative models since their introduction in Glow. Several attempts have been made to design invertible $k \times k$ convolutions that are efficient in training and sampling passes. Though these attempts have improved the expressivity and sampling efficiency, they severely lagged behind Glow which used only $1 \times 1$ convolutions in terms of sampling time. Also, many of the approaches mask a large number of parameters of the underlying convolution, resulting in lower expressivity on a fixed run-time budget. We propose a $k \times k$ convolutional layer and Deep Normalizing Flow architecture which i.) has a fast parallel inversion algorithm with running time O$(n k^2)$ ($n$ is height and width of the input image and k is kernel size), ii.) masks the minimal amount of learnable parameters in a layer. iii.) gives better forward pass and sampling times comparable to other $k \times k$ convolution-based models on real-world benchmarks. We provide an implementation of the proposed parallel algorithm for sampling using our invertible convolutions on GPUs. Benchmarks on CIFAR-10, ImageNet, and CelebA datasets show comparable performance to previous works regarding bits per dimension while significantly improving the sampling time.
InfiniCity: Infinite-Scale City Synthesis
Authors: Chieh Hubert Lin, Hsin-Ying Lee, Willi Menapace, Menglei Chai, Aliaksandr Siarohin, Ming-Hsuan Yang, Sergey Tulyakov
Date: 2023-01-23 00:00:00
Proceeding: ICCV 2023 1
Tasks: Image Generation, Neural Rendering

Toward infinite-scale 3D city synthesis, we propose a novel framework, InfiniCity, which constructs and renders an unconstrainedly large and 3D-grounded environment from random noises. InfiniCity decomposes the seemingly impractical task into three feasible modules, taking advantage of both 2D and 3D data. First, an infinite-pixel image synthesis module generates arbitrary-scale 2D maps from the bird's-eye view. Next, an octree-based voxel completion module lifts the generated 2D map to 3D octrees. Finally, a voxel-based neural rendering module texturizes the voxels and renders 2D images. InfiniCity can thus synthesize arbitrary-scale and traversable 3D city environments, and allow flexible and interactive editing from users. We quantitatively and qualitatively demonstrate the efficacy of the proposed framework. Project page: https://hubert0527.github.io/infinicity/
Prediction-Powered Inference
Authors: Anastasios N. Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I. Jordan, Tijana Zrnic
Date: 2023-01-23 00:00:00
Tasks: Astronomy, Prediction, regression, valid

Prediction-powered inference is a framework for performing valid statistical inference when an experimental dataset is supplemented with predictions from a machine-learning system. The framework yields simple algorithms for computing provably valid confidence intervals for quantities such as means, quantiles, and linear and logistic regression coefficients, without making any assumptions on the machine-learning algorithm that supplies the predictions. Furthermore, more accurate predictions translate to smaller confidence intervals. Prediction-powered inference could enable researchers to draw valid and more data-efficient conclusions using machine learning. The benefits of prediction-powered inference are demonstrated with datasets from proteomics, astronomy, genomics, remote sensing, census analysis, and ecology.
Feature construction using explanations of individual predictions
Authors: Boštjan Vouk, Matej Guid, Marko Robnik-Šikonja
Date: 2023-01-23 00:00:00
Tasks: Attribute

Feature construction can contribute to comprehensibility and performance of machine learning models. Unfortunately, it usually requires exhaustive search in the attribute space or time-consuming human involvement to generate meaningful features. We propose a novel heuristic approach for reducing the search space based on aggregation of instance-based explanations of predictive models. The proposed Explainable Feature Construction (EFC) methodology identifies groups of co-occurring attributes exposed by popular explanation methods, such as IME and SHAP. We empirically show that reducing the search to these groups significantly reduces the time of feature construction using logical, relational, Cartesian, numerical, and threshold num-of-N and X-of-N constructive operators. An analysis on 10 transparent synthetic datasets shows that EFC effectively identifies informative groups of attributes and constructs relevant features. Using 30 real-world classification datasets, we show significant improvements in classification accuracy for several classifiers and demonstrate the feasibility of the proposed feature construction even for large datasets. Finally, EFC generated interpretable features on a real-world problem from the financial industry, which were confirmed by a domain expert.
Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning
Authors: Malte Ostendorff, Georg Rehm
Date: 2023-01-23 00:00:00
Tasks: Cross-Lingual Transfer, Language Modeling, Language Modelling, Transfer Learning

Most Transformer language models are primarily pretrained on English text, limiting their use for other languages. As the model sizes grow, the performance gap between English and other languages with fewer compute and data resources increases even further. Consequently, more resource-efficient training methods are needed to bridge the gap for languages with fewer resources available. To address this problem, we introduce a cross-lingual and progressive transfer learning approach, called CLP-Transfer, that transfers models from a source language, for which pretrained models are publicly available, like English, to a new target language. As opposed to prior work, which focused on the cross-lingual transfer between two languages, we extend the transfer to the model size. Given a pretrained model in a source language, we aim for a same-sized model in a target language. Instead of training a model from scratch, we exploit a smaller model that is in the target language but requires much fewer resources. Both small and source models are then used to initialize the token embeddings of the larger model based on the overlapping vocabulary of the source and target language. All remaining weights are reused from the model in the source language. This approach outperforms the sole cross-lingual transfer and can save up to 80% of the training steps compared to the random initialization.
Asymptotic Convergence and Performance of Multi-Agent Q-Learning Dynamics
Authors: Aamal Abbas Hussain, Francesco Belardinelli, Georgios Piliouras
Date: 2023-01-23 00:00:00
Tasks: Q-Learning

Achieving convergence of multiple learning agents in general $N$-player games is imperative for the development of safe and reliable machine learning (ML) algorithms and their application to autonomous systems. Yet it is known that, outside the bounds of simple two-player games, convergence cannot be taken for granted. To make progress in resolving this problem, we study the dynamics of smooth Q-Learning, a popular reinforcement learning algorithm which quantifies the tendency for learning agents to explore their state space or exploit their payoffs. We show a sufficient condition on the rate of exploration such that the Q-Learning dynamics is guaranteed to converge to a unique equilibrium in any game. We connect this result to games for which Q-Learning is known to converge with arbitrary exploration rates, including weighted Potential games and weighted zero sum polymatrix games. Finally, we examine the performance of the Q-Learning dynamic as measured by the Time Averaged Social Welfare, and comparing this with the Social Welfare achieved by the equilibrium. We provide a sufficient condition whereby the Q-Learning dynamic will outperform the equilibrium even if the dynamics do not converge.
SpArX: Sparse Argumentative Explanations for Neural Networks [Technical Report]
Authors: Hamed Ayoobi, Nico Potyka, Francesca Toni
Date: 2023-01-23 00:00:00

Neural networks (NNs) have various applications in AI, but explaining their decisions remains challenging. Existing approaches often focus on explaining how changing individual inputs affects NNs' outputs. However, an explanation that is consistent with the input-output behaviour of an NN is not necessarily faithful to the actual mechanics thereof. In this paper, we exploit relationships between multi-layer perceptrons (MLPs) and quantitative argumentation frameworks (QAFs) to create argumentative explanations for the mechanics of MLPs. Our SpArX method first sparsifies the MLP while maintaining as much of the original structure as possible. It then translates the sparse MLP into an equivalent QAF to shed light on the underlying decision process of the MLP, producing global and/or local explanations. We demonstrate experimentally that SpArX can give more faithful explanations than existing approaches, while simultaneously providing deeper insights into the actual reasoning process of MLPs.
The Entoptic Field Camera as Metaphor-Driven Research-through-Design with AI Technologies
Authors: Jesse Josua Benjamin, Heidi Biggs, Arne Berger, Julija Rukanskaitė, Michael Heidt, Nick Merrill, James Pierce, Joseph Lindley
Date: 2023-01-23 00:00:00
Tasks: Image Generation

Artificial intelligence (AI) technologies are widely deployed in smartphone photography; and prompt-based image synthesis models have rapidly become commonplace. In this paper, we describe a Research-through-Design (RtD) project which explores this shift in the means and modes of image production via the creation and use of the Entoptic Field Camera. Entoptic phenomena usually refer to perceptions of floaters or bright blue dots stemming from the physiological interplay of the eye and brain. We use the term entoptic as a metaphor to investigate how the material interplay of data and models in AI technologies shapes human experiences of reality. Through our case study using first-person design and a field study, we offer implications for critical, reflective, more-than-human and ludic design to engage AI technologies; the conceptualisation of an RtD research space which contributes to AI literacy discourses; and outline a research trajectory concerning materiality and design affordances of AI technologies.