pwc-archive/papers-with-abstracts huggingface dataset:
XC-NAS: A New Cellular Encoding Approach for Neural Architecture Search of Multi-path Convolutional Neural Networks
Authors: Trevor Londt, Xiaoying Gao, Peter Andreae, Yi Mei
Date: 2023-12-12 00:00:00
Tasks: GPU, Neural Architecture Search, text-classification, Text Classification

Convolutional Neural Networks (CNNs) continue to achieve great success in classification tasks as innovative techniques and complex multi-path architecture topologies are introduced. Neural Architecture Search (NAS) aims to automate the design of these complex architectures, reducing the need for costly manual design work by human experts. Cellular Encoding (CE) is an evolutionary computation technique which excels in constructing novel multi-path topologies of varying complexity and has recently been applied with NAS to evolve CNN architectures for various classification tasks. However, existing CE approaches have severe limitations. They are restricted to only one domain, only partially implement the theme of CE, or only focus on the micro-architecture search space. This paper introduces a new CE representation and algorithm capable of evolving novel multi-path CNN architectures of varying depth, width, and complexity for image and text classification tasks. The algorithm explicitly focuses on the macro-architecture search space. Furthermore, by using a surrogate model approach, we show that the algorithm can evolve a performant CNN architecture in less than one GPU day, thereby allowing a sufficient number of experiment runs to be conducted to achieve scientific robustness. Experiment results show that the approach is highly competitive, defeating several state-of-the-art methods, and is generalisable to both the image and text domains.
Capacity of the treelike sign perceptrons neural networks with one hidden layer -- RDT based upper bounds
Authors: Mihailo Stojnic
Date: 2023-12-13 00:00:00

We study the capacity of \emph{sign} perceptrons neural networks (SPNN) and particularly focus on 1-hidden layer \emph{treelike committee machine} (TCM) architectures. Similarly to what happens in the case of a single perceptron neuron, it turns out that, in a statistical sense, the capacity of a corresponding multilayered network architecture consisting of multiple \emph{sign} perceptrons also undergoes the so-called phase transition (PT) phenomenon. This means: (i) for certain range of system parameters (size of data, number of neurons), the network can be properly trained to accurately memorize \emph{all} elements of the input dataset; and (ii) outside the region such a training does not exist. Clearly, determining the corresponding phase transition curve that separates these regions is an extraordinary task and among the most fundamental questions related to the performance of any network. Utilizing powerful mathematical engine called Random Duality Theory (RDT), we establish a generic framework for determining the upper bounds on the 1-hidden layer TCM SPNN capacity. Moreover, we do so for \emph{any} given (odd) number of neurons. We further show that the obtained results \emph{exactly} match the replica symmetry predictions of \cite{EKTVZ92,BHS92}, thereby proving that the statistical physics based results are not only nice estimates but also mathematically rigorous bounds as well. Moreover, for $d\leq 5$, we obtain the capacity values that improve on the best known rigorous ones of \cite{MitchDurb89}, thereby establishing a first, mathematically rigorous, progress in well over 30 years.
Leveraging User Simulation to Develop and Evaluate Conversational Information Access Agents
Authors: Nolwenn Bernard
Date: 2023-12-13 00:00:00
Tasks: Conversational Information Access, Reinforcement Learning, User Simulation

We observe a change in the way users access information, that is, the rise of conversational information access (CIA) agents. However, the automatic evaluation of these agents remains an open challenge. Moreover, the training of CIA agents is cumbersome as it mostly relies on conversational corpora, expert knowledge, and reinforcement learning. User simulation has been identified as a promising solution to tackle automatic evaluation and has been previously used in reinforcement learning. In this research, we investigate how user simulation can be leveraged in the context of CIA. We organize the work in three parts. We begin with the identification of requirements for user simulators for training and evaluating CIA agents and compare existing types of simulator regarding these. Then, we plan to combine these different types of simulators into a new hybrid simulator. Finally, we aim to extend simulators to handle more complex information seeking scenarios.
Quantum Private Information Retrieval from Coded Storage Systems
Authors: Matteo Allaix
Date: 2023-12-10 00:00:00
Tasks: Information Retrieval, Retrieval

In the era of extensive data growth, robust and efficient mechanisms are needed to store and manage vast amounts of digital information, such as Data Storage Systems (DSSs). Concurrently, privacy concerns have arisen, leading to the development of techniques like Private Information Retrieval (PIR) to enable data access while preserving privacy. A PIR protocol allows users to retrieve information from a database without revealing the specifics of their query or the data they are accessing. With the advent of quantum computing, researchers have explored the potential of using quantum systems to enhance privacy in information retrieval. In a Quantum Private Information Retrieval (QPIR) protocol, a user can retrieve information from a database by downloading quantum systems from multiple servers, while ensuring that the servers remain oblivious to the specific information being accessed. This scenario offers a unique advantage by leveraging the inherent properties of quantum systems to provide enhanced privacy guarantees and improved communication rates compared to classical PIR protocols. In this thesis we consider the QPIR setting where the queries and the coded storage systems are classical, while the responses from the servers are quantum. This problem was treated by Song et al. for replicated storage and different collusion patterns. This thesis aims to develop QPIR protocols for coded storage by combining known classical PIR protocols with quantum communication algorithms, achieving enhanced privacy and communication costs. We consider different storage codes and robustness assumptions, and we prove that the achieved communication cost is always lower than the classical counterparts.
Federated Learning for Short Text Clustering
Authors: Mengling Hu, Chaochao Chen, Weiming Liu, Xinting Liao, Xiaolin Zheng
Date: 2023-11-23 00:00:00
Tasks: Clustering, Federated Learning, Short Text Clustering, Text Clustering

Short text clustering has been popularly studied for its significance in mining valuable insights from many short texts. In this paper, we focus on the federated short text clustering (FSTC) problem, i.e., clustering short texts that are distributed in different clients, which is a realistic problem under privacy requirements. Compared with the centralized short text clustering problem that short texts are stored on a central server, the FSTC problem has not been explored yet. To fill this gap, we propose a Federated Robust Short Text Clustering (FSTC) framework. FSTC includes two main modules, i.e., robust short text clustering module and federated cluster center aggregation module. The robust short text clustering module aims to train an effective short text clustering model with local data in each client. We innovatively combine optimal transport to generate pseudo-labels with Gaussian-uniform mixture model to ensure the reliability of the pseudo-supervised data. The federated cluster center aggregation module aims to exchange knowledge across clients without sharing local raw data in an efficient way. The server aggregates the local cluster centers from different clients and then sends the global centers back to all clients in each communication round. Our empirical studies on three short text clustering datasets demonstrate that FSTC significantly outperforms the federated short text clustering baselines.
Fast k-NN Graph Construction by GPU based NN-Descent
Authors: Hui Wang, Wan-Lei Zhao, Xiangxiang Zeng, Jianye Yang
Date: 2021-10-30 00:00:00
Proceeding: CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management 2021 10
Tasks: CPU, GPU, graph construction, Information Retrieval, Retrieval

NN-Descent is a classic k-NN graph construction approach. It is still widely employed in machine learning, computer vision, and information retrieval tasks due to its efficiency and genericness. However, the current design only works well on CPU. In this paper, NN-Descent has been redesigned to adapt to the GPU architecture. A new graph update strategy called selective update is proposed. It reduces the data exchange between GPU cores and GPU global memory significantly, which is the processing bottleneck under GPU computation architecture. This redesign leads to full exploitation of the parallelism of the GPU hardware. In the meantime, the genericness, as well as the simplicity of NN-Descent, are well-preserved. Moreover, a procedure that allows to k-NN graph to be merged efficiently on GPU is proposed. It makes the construction of high-quality k-NN graphs for out-of-GPU-memory datasets tractable. Our approach is 100-250× faster than the single-thread NN-Descent and is 2.5-5× faster than the existing GPU-based approaches as we tested on million as well as billion scale datasets.
Graph-based prediction of Protein-protein interactions with attributed signed graph embedding
Authors: Fang Yang, Kunjie Fan, Dandan song, Huakang Lin
Date: 2020-07-21 00:00:00
Proceeding: BMC Bioinformatics 2020 7
Tasks: Graph Embedding, Graph Representation Learning, Representation Learning

Background
Protein-protein interactions (PPIs) are central to many biological processes. Considering that the experimental methods for identifying PPIs are time-consuming and expensive, it is important to develop automated computational methods to better predict PPIs. Various machine learning methods have been proposed, including a deep learning technique which is sequence-based that has achieved promising results. However, it only focuses on sequence information while ignoring the structural information of PPI networks. Structural information of PPI networks such as their degree, position, and neighboring nodes in a graph has been proved to be informative in PPI prediction.

Results
Facing the challenge of representing graph information, we introduce an improved graph representation learning method. Our model can study PPI prediction based on both sequence information and graph structure. Moreover, our study takes advantage of a representation learning model and employs a graph-based deep learning method for PPI prediction, which shows superiority over existing sequence-based methods. Statistically, Our method achieves state-of-the-art accuracy of 99.15% on Human protein reference database (HPRD) dataset and also obtains best results on Database of Interacting Protein (DIP) Human, Drosophila, Escherichia coli (E. coli), and Caenorhabditis elegans (C. elegan) datasets.

Conclusion
Here, we introduce signed variational graph auto-encoder (S-VGAE), an improved graph representation learning method, to automatically learn to encode graph structure into low-dimensional embeddings. Experimental results demonstrate that our method outperforms other existing sequence-based methods on several datasets. We also prove the robustness of our model for very sparse networks and the generalization for a new dataset that consists of four datasets: HPRD, E.coli, C.elegan, and Drosophila.
spateGAN: Spatio-Temporal Downscaling of Rainfall Fields Using a cGAN Approach
Authors: Luca Glawion, Julius Polz, Harald Kunstmann, Benjamin Fersch, Christian Chwala
Date: 2023-10-16 00:00:00
Proceeding: Earth and Space Science 2023 10
Tasks: Computational Efficiency, Model Selection, Super-Resolution, Video Super-Resolution

Climate models face limitations in their ability to accurately represent highly variable atmospheric phenomena. To resolve fine-scale physical processes, allowing for local impact assessments, downscaling techniques are essential. We propose spateGAN, a novel approach for spatio-temporal downscaling of precipitation data using conditional generative adversarial networks. Our method is based on a video super-resolution approach and trained on 10 years of country-wide radar observations for Germany. It simultaneously increases the spatial and temporal resolution of coarsened precipitation observations from 32 to 2 km and from 1 hr to 10 min. Our experiments indicate that the ensembles of generated temporally consistent rainfall fields are in high agreement with the observational data. Spatial structures with plausible advection were accurately generated. Compared to trilinear interpolation and a classical convolutional neural network, the generative model reconstructs the resolution-dependent extreme value distribution with high skill. It showed a high fractions skill score of 0.6 (spatio-temporal scale: 32 km and 1 hr) for rainfall intensities over 15 mm h−1 and a low relative bias of 3.35%. A power spectrum analysis confirmed that the probabilistic downscaling ability of our model further increased its skill. We observed that neural network predictions may be interspersed by recurrent structures not related to rainfall climatology, which should be a known issue for future studies. We were able to mitigate them by using an appropriate model architecture and model selection process. Our findings suggest that spateGAN offers the potential to complement and further advance the development of climate model downscaling techniques, due to its performance and computational efficiency.
PRGAN: Personalized Recommendation with Conditional Generative Adversarial Networks
Authors: Jing Wen, Bi-Yi Chen, Chang-Dong Wang, Zhihong Tian
Date: 2021-07-10 00:00:00
Proceeding: 2021 IEEE International Conference on Data Mining (ICDM) 2021 7
Tasks: Data Augmentation, Generative Adversarial Network, Recommendation Systems

Most of the existing methods define recommendation as regression or classification for user-item interactions and apply discriminative models. However, recommender systems suffer from interaction data sparsity and data noise problems in reality. Recent Generative Adversarial Network-based recommender systems have the potential to solve the aforementioned problems. The negative sampling methods use the generator to collect effective signals from a large amount of unlabeled data to alleviate the data sparsity problem, while they suffer from sparse rewards in the policy gradient training process. The vector reconstruction methods generate user-related vectors for data augmentation to enhance robustness, but they lead to redundant calculation and only take the user as a condition and ignore information conveyed by items. To alleviate the limitations of these methods, we propose a novel framework termed Personalized Recommendation with Conditional Generative Adversarial Networks (PRGAN) to consider both the user and the item subset as conditions and formulate conditional rating vector generation as a user-item matching problem. The sparsity of conditional rating vectors can be controlled in our method, which simplifies the discriminator’s learning task. Experiments are conducted on four datasets to evaluate the effectiveness of the proposed framework.
ArcGAN: Generative Adversarial Networks for 3D Architectural Image Generation
Authors: Drashty Ranpara, Raghul V, Arun Mozhi Devan, Dr. Samiksha Shukla
Date: 2023-12-14 00:00:00
Proceeding: IDEA 2K22 2023 12
Tasks: 3D Architecture, 3D Depth Estimation, 3D Semantic Scene Completion from a single 2D image, 3D Shape Modeling, 3D Shape Reconstruction From A Single 2D Image, 3D Shape Representation, Conditional Image Generation, Generative Adversarial Network, Image Generation, Multi-View 3D Shape Retrieval

Due to advancements in infrastructural modulations, architectural design is one of the most peculiar and tedious processes. As the technology evolves to the next phase, using some latest techniques like generative adversarial networks, creating a hybrid architectural design from old and new models is possible with maximum accuracy. Training the model with appropriate samples makes it evident that the designing phase will be simple for even a layman by including proper parameters such as material description, structural engineering, etc. This research paper suggests a hybrid model for an architectural design using generative adversarial networks. For example, merging Rome's architectural style with Italy's will accurately and precisely recover the pixel-level structure of 3D forms without needing a 2D viewpoint or 3D annotations from a real 2D generated image.
Annotating sleep states in children from wrist-worn accelerometer data using Machine Learning
Authors: Ashwin Ram, Sundar Sripada V. S., Shuvam Keshari, Zizhe Jiang
Date: 2023-12-09 00:00:00
Tasks: Event Detection

Sleep detection and annotation are crucial for researchers to understand sleep patterns, especially in children. With modern wrist-worn watches comprising built-in accelerometers, sleep logs can be collected. However, the annotation of these logs into distinct sleep events: onset and wakeup, proves to be challenging. These annotations must be automated, precise, and scalable. We propose to model the accelerometer data using different machine learning (ML) techniques such as support vectors, boosting, ensemble methods, and more complex approaches involving LSTMs and Region-based CNNs. Later, we aim to evaluate these approaches using the Event Detection Average Precision (EDAP) score (similar to the IOU metric) to eventually compare the predictive power and model performance.
Efficient Training of Energy-Based Models Using Jarzynski Equality
Authors: Davide Carbone, Mengjian Hua, Simon Coste, Eric Vanden-Eijnden
Date: 2023-05-30 00:00:00
Proceeding: NeurIPS 2023 11

Energy-based models (EBMs) are generative models inspired by statistical physics with a wide range of applications in unsupervised learning. Their performance is best measured by the cross-entropy (CE) of the model distribution relative to the data distribution. Using the CE as the objective for training is however challenging because the computation of its gradient with respect to the model parameters requires sampling the model distribution. Here we show how results for nonequilibrium thermodynamics based on Jarzynski equality together with tools from sequential Monte-Carlo sampling can be used to perform this computation efficiently and avoid the uncontrolled approximations made using the standard contrastive divergence algorithm. Specifically, we introduce a modification of the unadjusted Langevin algorithm (ULA) in which each walker acquires a weight that enables the estimation of the gradient of the cross-entropy at any step during GD, thereby bypassing sampling biases induced by slow mixing of ULA. We illustrate these results with numerical experiments on Gaussian mixture distributions as well as the MNIST dataset. We show that the proposed approach outperforms methods based on the contrastive divergence algorithm in all the considered situations.
Laboratory test results underwater navigation system for environmental control devices
Authors: A. N. Grekov, M. A Pasynkov, . S. S. Peliushenko
Date: 2023-12-13 00:00:00

The work analyzes existing methods for increasing the accuracy of determining weekend navigation parameters of unmanned underwater vehicles. It is proposed to retrofit the navigation system with an additional hydrodynamic tilt unit, which will increase accuracy of coordinate determination. A prototype of the proposed system was made, and algorithmic software for it. Analysis of typical MEMS errors has been completed accelerometers and gyroscopes. To estimate stochastic errors, the Allan variation method was tested using a set of real data obtained on a prototype as input parameters. A model of stochastic errors of all measuring channels has been constructed
Preliminary Guidelines for Electrode Positioning in Noninvasive Deep Brain Stimulation via Temporally Interfering Electric Fields
Authors: Mobina Zibandepour, Akram Shojaei, Arash Abbasi Larki, Mehdi Delrobaei
Date: 2023-12-13 00:00:00

Advancements in neurosurgical robotics have improved medical procedures, particularly deep brain stimulation, where robots combine human and machine intelligence to precisely implant electrodes in the brain. While effective, this procedure carries risks and side effects. Noninvasive deep brain stimulation (NIDBS) offers promise by making brain stimulation safer, more affordable, and accessible. However, NIDBS lacks guidelines for electrode placement. This study explores adapting robotic principles to enhance the accuracy of NIDBS targeting and provides preliminary guidelines for transcranial electrode placement. Safety is also emphasized, ensuring a balance between therapeutic effectiveness and patient safety by maintaining electric fields within safe limits.
Provably-Correct Safety Protocol for Cooperative Platooning
Authors: Sebastian Mair, Matthias Althoff
Date: 2023-12-13 00:00:00
Tasks: Collision Avoidance

Cooperative Adaptive Cruise Control (CACC) is a well-studied technology for forming string-stable vehicle platoons. Ensuring collision avoidance is particularly difficult in CACC due to the small desired inter-vehicle spacing. We propose a safety protocol preventing collisions in a provably-correct manner while still maintaining a small distance to the preceding vehicle, by utilizing communicated braking capabilities. In addition, the safety of the protocol is ensured despite possible communication failures. While our concept can be applied to any CACC system, we particularly consider a class of CACCs, where the platoon vehicles successively agree on a consensus behavior. Our safety protocol is evaluated on various scenarios using the CommonRoad benchmark suite.
Event-Triggered Safe Bayesian Optimization on Quadcopters
Authors: Antonia Holzapfel, Paul Brunzema, Sebastian Trimpe
Date: 2023-12-13 00:00:00
Tasks: Bayesian Optimization

Bayesian optimization (BO) has proven to be a powerful tool for automatically tuning control parameters without requiring knowledge of the underlying system dynamics. Safe BO methods, in addition, guarantee safety during the optimization process, assuming that the underlying objective function does not change. However, in real-world scenarios, time-variations frequently occur, for example, due to wear in the system or changes in operation. Utilizing standard safe BO strategies that do not address time-variations can result in failure as previous safe decisions may become unsafe over time, which we demonstrate herein. To address this, we introduce a new algorithm, Event-Triggered SafeOpt (ETSO), which adapts to changes online solely relying on the observed costs. At its core, ETSO uses an event trigger to detect significant deviations between observations and the current surrogate of the objective function. When such change is detected, the algorithm reverts to a safe backup controller, and exploration is restarted. In this way, safety is recovered and maintained across changes. We evaluate ETSO on quadcopter controller tuning, both in simulation and hardware experiments. ETSO outperforms state-of-the-art safe BO, achieving superior control performance over time while maintaining safety.