pwc-archive/papers-with-abstracts huggingface dataset:
How to Do Things with Deep Learning Code
Authors: Minh Hua, Rita Raley
Date: 2023-04-19 00:00:00
Tasks: Deep Learning

The premise of this article is that a basic understanding of the composition and functioning of large language models is critically urgent. To that end, we extract a representational map of OpenAI's GPT-2 with what we articulate as two classes of deep learning code, that which pertains to the model and that which underwrites applications built around the model. We then verify this map through case studies of two popular GPT-2 applications: the text adventure game, AI Dungeon, and the language art project, This Word Does Not Exist. Such an exercise allows us to test the potential of Critical Code Studies when the object of study is deep learning code and to demonstrate the validity of code as an analytical focus for researchers in the subfields of Critical Artificial Intelligence and Critical Machine Learning Studies. More broadly, however, our work draws attention to the means by which ordinary users might interact with, and even direct, the behavior of deep learning systems, and by extension works toward demystifying some of the auratic mystery of "AI." What is at stake is the possibility of achieving an informed sociotechnical consensus about the responsible applications of large language models, as well as a more expansive sense of their creative capabilities-indeed, understanding how and where engagement occurs allows all of us to become more active participants in the development of machine learning systems.
The Unintended Consequences of Censoring Digital Technology -- Evidence from Italy's ChatGPT Ban
Authors: David H. Kreitmeir, Paul A. Raschky
Date: 2023-04-18 00:00:00
Tasks: Chatbot

We analyse the effects of the ban of ChatGPT, a generative pre-trained transformer chatbot, on individual productivity. We first compile data on the hourly coding output of over 8,000 professional GitHub users in Italy and other European countries to analyse the impact of the ban on individual productivity. Combining the high-frequency data with the sudden announcement of the ban in a difference-in-differences framework, we find that the output of Italian developers decreased by around 50% in the first two business days after the ban and recovered after that. Applying a synthetic control approach to daily Google search and Tor usage data shows that the ban led to a significant increase in the use of censorship bypassing tools. Our findings show that users swiftly implement strategies to bypass Internet restrictions but this adaptation activity creates short-term disruptions and hampers productivity.
BIM-GPT: a Prompt-Based Virtual Assistant Framework for BIM Information Retrieval
Authors: Junwen Zheng, Martin Fischer
Date: 2023-04-18 00:00:00
Tasks: Information Retrieval, Retrieval

Efficient information retrieval (IR) from building information models (BIMs) poses significant challenges due to the necessity for deep BIM knowledge or extensive engineering efforts for automation. We introduce BIM-GPT, a prompt-based virtual assistant (VA) framework integrating BIM and generative pre-trained transformer (GPT) technologies to support NL-based IR. A prompt manager and dynamic template generate prompts for GPT models, enabling interpretation of NL queries, summarization of retrieved information, and answering BIM-related questions. In tests on a BIM IR dataset, our approach achieved 83.5% and 99.5% accuracy rates for classifying NL queries with no data and 2% data incorporated in prompts, respectively. Additionally, we validated the functionality of BIM-GPT through a VA prototype for a hospital building. This research contributes to the development of effective and versatile VAs for BIM IR in the construction industry, significantly enhancing BIM accessibility and reducing engineering efforts and training data requirements for processing NL queries.
Genetic Algorithm Based Combinatorial Optimization for the Optimal Design of Water Distribution Network of Gurudeniya Service Zone, Sri Lanka
Authors: K. H. M. R. N. Senavirathna, C. K. Walgampaya
Date: 2023-04-19 00:00:00
Tasks: Combinatorial Optimization

This paper brings an in detail Genetic Algorithm (GA) based combinatorial optimization method used for the optimal design of the water distribution network (WDN) of Gurudeniya Service Zone, Sri Lanka. Genetic Algorithm (GA) mimics the survival of the fittest principle of nature to develop a search process. Methodology employs fuzzy combinations of pipe diameters to check their suitability to be considered as the cost effective optimal design solutions. Furthermore, the hydraulic constraints were implicitly evaluated within the GA itself in its aim to reaching the global optimum solution. Upon analysis, the results of this approach delivered agreeable design outputs. In addition, the comparison made between the results obtained by a previous study inspired by the Honey Bee Mating Optimization (HBMO) Algorithm and results obtained by the GA based approach, proves competency of GA for the optimal design of water distribution network in Gurudeniya Service Zone, Sri Lanka.
Comma Selection Outperforms Plus Selection on OneMax with Randomly Planted Optima
Authors: Joost Jorritsma, Johannes Lengler, Dirk Sudholt
Date: 2023-04-19 00:00:00
Tasks: Evolutionary Algorithms

It is an ongoing debate whether and how comma selection in evolutionary algorithms helps to escape local optima. We propose a new benchmark function to investigate the benefits of comma selection: OneMax with randomly planted local optima, generated by frozen noise. We show that comma selection (the $(1,\lambda)$ EA) is faster than plus selection (the $(1+\lambda)$ EA) on this benchmark, in a fixed-target scenario, and for offspring population sizes $\lambda$ for which both algorithms behave differently. For certain parameters, the $(1,\lambda)$ EA finds the target in $\Theta(n \ln n)$ evaluations, with high probability (w.h.p.), while the $(1+\lambda)$ EA) w.h.p. requires almost $\Theta((n\ln n)^2)$ evaluations. We further show that the advantage of comma selection is not arbitrarily large: w.h.p. comma selection outperforms plus selection at most by a factor of $O(n \ln n)$ for most reasonable parameter choices. We develop novel methods for analysing frozen noise and give powerful and general fixed-target results with tail bounds that are of independent interest.
Analysing Equilibrium States for Population Diversity
Authors: Johannes Lengler, Andre Opris, Dirk Sudholt
Date: 2023-04-19 00:00:00
Tasks: Diversity, Evolutionary Algorithms

Population diversity is crucial in evolutionary algorithms as it helps with global exploration and facilitates the use of crossover. Despite many runtime analyses showing advantages of population diversity, we have no clear picture of how diversity evolves over time. We study how population diversity of $(\mu+1)$ algorithms, measured by the sum of pairwise Hamming distances, evolves in a fitness-neutral environment. We give an exact formula for the drift of population diversity and show that it is driven towards an equilibrium state. Moreover, we bound the expected time for getting close to the equilibrium state. We find that these dynamics, including the location of the equilibrium, are unaffected by surprisingly many algorithmic choices. All unbiased mutation operators with the same expected number of bit flips have the same effect on the expected diversity. Many crossover operators have no effect at all, including all binary unbiased, respectful operators. We review crossover operators from the literature and identify crossovers that are neutral towards the evolution of diversity and crossovers that are not.
Parallel Neural Networks in Golang
Authors: Daniela Kalwarowskyj, Erich Schikuta
Date: 2023-04-19 00:00:00
Tasks: Neural Network simulation

This paper describes the design and implementation of parallel neural networks (PNNs) with the novel programming language Golang. We follow in our approach the classical Single-Program Multiple-Data (SPMD) model where a PNN is composed of several sequential neural networks, which are trained with a proportional share of the training dataset. We used for this purpose the MNIST dataset, which contains binary images of handwritten digits. Our analysis focusses on different activation functions and optimizations in the form of stochastic gradients and initialization of weights and biases. We conduct a thorough performance analysis, where network configurations and different performance factors are analyzed and interpreted. Golang and its inherent parallelization support proved very well for parallel neural network simulation by considerable decreased processing times compared to sequential variants.
Learning Hierarchically-Structured Concepts II: Overlapping Concepts, and Networks With Feedback
Authors: Nancy Lynch, Frederik Mallmann-Trenn
Date: 2023-04-19 00:00:00

We continue our study from Lynch and Mallmann-Trenn (Neural Networks, 2021), of how concepts that have hierarchical structure might be represented in brain-like neural networks, how these representations might be used to recognize the concepts, and how these representations might be learned. In Lynch and Mallmann-Trenn (Neural Networks, 2021), we considered simple tree-structured concepts and feed-forward layered networks. Here we extend the model in two ways: we allow limited overlap between children of different concepts, and we allow networks to include feedback edges. For these more general cases, we describe and analyze algorithms for recognition and algorithms for learning.
Modular Differential Evolution
Authors: Diederick Vermetten, Fabio Caraffini, Anna V. Kononova, Thomas Bäck
Date: 2023-04-19 00:00:00

New contributions in the field of iterative optimisation heuristics are often made in an iterative manner. Novel algorithmic ideas are not proposed in isolation, but usually as an extension of a preexisting algorithm. Although these contributions are often compared to the base algorithm, it is challenging to make fair comparisons between larger sets of algorithm variants. This happens because even small changes in the experimental setup, parameter settings, or implementation details can cause results to become incomparable. Modular algorithms offer a way to overcome these challenges. By implementing the algorithmic modifications into a common framework, many algorithm variants can be compared, while ensuring that implementation details match in all versions. In this work, we propose a version of a modular framework for the popular Differential Evolution (DE) algorithm. We show that this modular approach not only aids in comparison, but also allows for a much more detailed exploration of the space of possible DE variants. This is illustrated by showing that tuning the settings of modular DE vastly outperforms a set of commonly used DE versions which have been recreated in our framework. We then investigate these tuned algorithms in detail, highlighting the relation between modules and performance on particular problems.
Minimax Signal Detection in Sparse Additive Models
Authors: Subhodh Kotekal, Chao GAO
Date: 2023-04-19 00:00:00
Tasks: Additive models

Sparse additive models are an attractive choice in circumstances calling for modelling flexibility in the face of high dimensionality. We study the signal detection problem and establish the minimax separation rate for the detection of a sparse additive signal. Our result is nonasymptotic and applicable to the general case where the univariate component functions belong to a generic reproducing kernel Hilbert space. Unlike the estimation theory, the minimax separation rate reveals a nontrivial interaction between sparsity and the choice of function space. We also investigate adaptation to sparsity and establish an adaptive testing rate for a generic function space; adaptation is possible in some spaces while others impose an unavoidable cost. Finally, adaptation to both sparsity and smoothness is studied in the setting of Sobolev space, and we correct some existing claims in the literature.
Visualising Personal Data Flows: Insights from a Case Study of Booking.com
Authors: Haiyue Yuan, Matthew Boakes, Xiao Ma, Dongmei Cao, Shujun Li
Date: 2023-04-19 00:00:00

Commercial organisations are holding and processing an ever-increasing amount of personal data. Policies and laws are continually changing to require these companies to be more transparent regarding the collection, storage, processing and sharing of this data. This paper reports our work of taking Booking.com as a case study to visualise personal data flows extracted from their privacy policy. By showcasing how the company shares its consumers' personal data, we raise questions and extend discussions on the challenges and limitations of using privacy policies to inform online users about the true scale and the landscape of personal data flows. This case study can inform us about future research on more data flow-oriented privacy policy analysis and on the construction of a more comprehensive ontology on personal data flows in complicated business ecosystems.
An Offline Metric for the Debiasedness of Click Models
Authors: Romain Deffayet, Philipp Hager, Jean-Michel Renders, Maarten de Rijke
Date: 2023-04-19 00:00:00
Tasks: counterfactual, Learning-To-Rank, Model Selection

A well-known problem when learning from user clicks are inherent biases prevalent in the data, such as position or trust bias. Click models are a common method for extracting information from user clicks, such as document relevance in web search, or to estimate click biases for downstream applications such as counterfactual learning-to-rank, ad placement, or fair ranking. Recent work shows that the current evaluation practices in the community fail to guarantee that a well-performing click model generalizes well to downstream tasks in which the ranking distribution differs from the training distribution, i.e., under covariate shift. In this work, we propose an evaluation metric based on conditional independence testing to detect a lack of robustness to covariate shift in click models. We introduce the concept of debiasedness in click modeling and derive a metric for measuring it. In extensive semi-synthetic experiments, we show that our proposed metric helps to predict the downstream performance of click models under covariate shift and is useful in an off-policy model selection setting.
Leveraging Deep Learning Techniques on Collaborative Filtering Recommender Systems
Authors: Ali Fallahi RahmatAbadi, Javad Mohammadzadeh
Date: 2023-04-18 00:00:00
Tasks: Collaborative Filtering, Deep Learning, Information Retrieval, Recommendation Systems, Retrieval

With the exponentially increasing volume of online data, searching and finding required information have become an extensive and time-consuming task. Recommender Systems as a subclass of information retrieval and decision support systems by providing personalized suggestions helping users access what they need more efficiently. Among the different techniques for building a recommender system, Collaborative Filtering (CF) is the most popular and widespread approach. However, cold start and data sparsity are the fundamental challenges ahead of implementing an effective CF-based recommender. Recent successful developments in enhancing and implementing deep learning architectures motivated many studies to propose deep learning-based solutions for solving the recommenders' weak points. In this research, unlike the past similar works about using deep learning architectures in recommender systems that covered different techniques generally, we specifically provide a comprehensive review of deep learning-based collaborative filtering recommender systems. This in-depth filtering gives a clear overview of the level of popularity, gaps, and ignored areas on leveraging deep learning techniques to build CF-based systems as the most influential recommenders.
Frequency Enhanced Hybrid Attention Network for Sequential Recommendation
Authors: Xinyu Du, Huanhuan Yuan, Pengpeng Zhao, Jianfeng Qu, Fuzhen Zhuang, Guanfeng Liu, Victor S. Sheng
Date: 2023-04-18 00:00:00
Tasks: Contrastive Learning, Sequential Recommendation

The self-attention mechanism, which equips with a strong capability of modeling long-range dependencies, is one of the extensively used techniques in the sequential recommendation field. However, many recent studies represent that current self-attention based models are low-pass filters and are inadequate to capture high-frequency information. Furthermore, since the items in the user behaviors are intertwined with each other, these models are incomplete to distinguish the inherent periodicity obscured in the time domain. In this work, we shift the perspective to the frequency domain, and propose a novel Frequency Enhanced Hybrid Attention Network for Sequential Recommendation, namely FEARec. In this model, we firstly improve the original time domain self-attention in the frequency domain with a ramp structure to make both low-frequency and high-frequency information could be explicitly learned in our approach. Moreover, we additionally design a similar attention mechanism via auto-correlation in the frequency domain to capture the periodic characteristics and fuse the time and frequency level attention in a union model. Finally, both contrastive learning and frequency regularization are utilized to ensure that multiple views are aligned in both the time domain and frequency domain. Extensive experiments conducted on four widely used benchmark datasets demonstrate that the proposed model performs significantly better than the state-of-the-art approaches.
Low-Dose CT Image Reconstruction using Vector Quantized Convolutional Autoencoder with Perceptual Loss
Authors: Shalini Ramanathan, Mohan Ramasundaram
Date: 2023-03-28 00:00:00
Proceeding: Sādhanā 2023 3
Tasks: Computed Tomography (CT), Image Reconstruction, Quantization

Computed Tomography (CT) has become a useful screening procedure to identify disease or injury within various regions of the human body. The human beings’ health issues caused by CT radiation have attracted the interest of the researchers and academic community. Reducing the radiation dose is the solution, but the CT image generated with low-dose radiation results in excessive noise due to lower intensity and fewer angle measurements. Low-dose CT scan images reduce image quality and thus affect a doctor’s diagnosis. Deep learning methods have become increasingly popular in recent years, many models have been proposed for Low-Dose CT image reconstruction. Low-Dose CT Image Reconstruction is an active area of modern medical imaging research. Deep learning-based medical image reconstruction methods will be helpful to reduce noise without compromising image quality. Therefore, this paper introduces a novel CT image reconstruction method based on the vector quantization technique utilized in the convolutional autoencoder network. The quality of the results is evaluated based on the perceptual loss function. Experimental evaluations are conducted on the LoDoPaB-CT benchmark dataset. Its result showed that the proposed network obtained better performance metric values and better noise elimination results, in terms of quantitative and visual evaluation, respectively.
Phase-Retrieval with Incomplete Autocorrelations Using Deep Convolutional Autoencoders
Authors: Giovanni Pellegrini, Jacopo Bertolotti
Date: 2023-04-18 00:00:00
Tasks: Retrieval

Phase-retrieval techniques aim to recover the original signal from just the modulus of its Fourier transform, which is usually much easier to measure than its phase, but the standard iterative techniques tend to fail if only part of the modulus information is available. We show that a neural network can be trained to perform phase retrieval using only incomplete information, and we discuss advantages and limitations of this approach.
Kernel independent component analysis
Authors: F.R. Bach; M.I. Jordan
Date: 2002-07-03 00:00:00
Proceeding: Journal of Machine Learning Research 2002 7

We present a class of algorithms for independent component analysis (ICA) which use contrast functions based on canonical correlations in a reproducing kernel Hilbert space. On the one hand, we show that our contrast functions are related to mutual information and have desirable mathematical properties as measures of statistical dependence. On the other hand, building on recent developments in kernel methods, we show that these criteria can be computed efficiently. Minimizing these criteria leads to flexible and robust algorithms for ICA. We illustrate with simulations involving a wide variety of source distributions, showing that our algorithms outperform many of the presently known algorithms.
Exploring novel algorithms for atrial fibrillation detection by driving graduate level education in medical machine learning
Authors: Maurice Rohr, Christoph Reich, Andreas Höhl, Timm Lilienthal, Tizian Dege, Filip Plesinger, Veronika Bulkova, Gari Clifford, Matthew Reyna, Christoph Hoog Antink
Date: 2022-07-07 00:00:00
Proceeding: Physiological Measurement 2022 7
Tasks: Atrial Fibrillation Detection

During the lockdown of universities and the COVID-Pandemic most students were restricted to their homes. Novel and instigating teaching methods were required to improve the learning experience and so recent implementations of the annual PhysioNet/Computing in Cardiology (CinC) Challenges posed as a reference. For over 20 years, the challenges have proven repeatedly to be of immense educational value, besides leading to technological advances for specific problems. In this paper, we report results from the class 'Artificial Intelligence in Medicine Challenge', which was implemented as an online project seminar at Technical University Darmstadt, Germany, and which was heavily inspired by the PhysioNet/CinC Challenge 2017 'AF Classification from a Short Single Lead ECG Recording'. Atrial fibrillation is a common cardiac disease and often remains undetected. Therefore, we selected the two most promising models of the course and give an insight into the Transformer-based DualNet architecture as well as into the CNN-LSTM-based model and finally a detailed analysis for both. In particular, we show the model performance results of our internal scoring process for all submitted models and the near state-of-the-art model performance for the two named models on the official 2017 challenge test set. Several teams were able to achieve F1 scores above/close to 90% on a hidden test-set of Holter recordings. We highlight themes commonly observed among participants, and report the results from the self-assessed student evaluation. Finally, the self-assessment of the students reported a notable increase in machine learning knowledge.
NOMARO: Defending against Adversarial Attacks by NOMA-Inspired Reconstruction Operation
Authors: Aryaman Sinha, Soumya P. Dash, N. B. Puhan
Date: 2021-12-14 00:00:00
Proceeding: IEEE Sensors Letters 2021 12
Tasks: Adversarial Attack, Adversarial Defense

In this work, a non-orthogonal multiple access (NOMA)-inspired defense method is proposed to mitigate the effect of adversarial attacks, which pose a major challenge towards deep neural networks (DNNs) in multimedia networks. The novel defense method, namely NOMA-inspired reconstruction operation (NOMARO), incorporates a copy of the input image generated by applying the untargeted adversarial attack. The copy and input images are superposed with a power allocation factor inversely proportional to the correlation between the considered images. To the best of our knowledge, this is the first communication theory based approach to design an adversarial defense method to be useful in multimedia applications. A comparative study with the existing defense techniques shows the superior performance of the proposed NOMARO defense against the state-of-the-art C&W and Square attacks in white-box and black-box settings, respectively, on popular DNN models.