pwc-archive/papers-with-abstracts huggingface dataset:
Neural Echo State Network using oscillations of gas bubbles in water
Authors: Ivan S. Maksymov, Andrey Pototsky, Sergey A. Suslov
Date: 2021-12-22 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

In the framework of physical reservoir computing (RC), machine learning algorithms designed for digital computers are executed using analog computer-like nonlinear physical systems that can provide energy-efficient computational power for predicting time-dependent quantities that can be found using nonlinear differential equations. We suggest a bubble-based RC (BRC) system that combines the nonlinearity of an acoustic response of a cluster of oscillating gas bubbles in water with a standard Echo State Network (ESN) algorithm that is well-suited to forecast chaotic time series. We confirm the plausibility of the BRC system by numerically demonstrating its ability to forecast certain chaotic time series similarly to or even more accurately than ESN.
Stochastic Continuous Submodular Maximization: Boosting via Non-oblivious Function
Authors: Qixin Zhang, Zengde Deng, Zaiyi Chen, Haoyuan Hu, Yu Yang
Date: 2022-01-03 00:00:00

In this paper, we revisit Stochastic Continuous Submodular Maximization in both offline and online settings, which can benefit wide applications in machine learning and operations research areas. We present a boosting framework covering gradient ascent and online gradient ascent. The fundamental ingredient of our methods is a novel non-oblivious function $F$ derived from a factor-revealing optimization problem, whose any stationary point provides a $(1-e^{-\gamma})$-approximation to the global maximum of the $\gamma$-weakly DR-submodular objective function $f\in C^{1,1}_L(\mathcal{X})$. Under the offline scenario, we propose a boosting gradient ascent method achieving $(1-e^{-\gamma}-\epsilon^{2})$-approximation after $O(1/\epsilon^2)$ iterations, which improves the $(\frac{\gamma^2}{1+\gamma^2})$ approximation ratio of the classical gradient ascent algorithm. In the online setting, for the first time we consider the adversarial delays for stochastic gradient feedback, under which we propose a boosting online gradient algorithm with the same non-oblivious function $F$. Meanwhile, we verify that this boosting online algorithm achieves a regret of $O(\sqrt{D})$ against a $(1-e^{-\gamma})$-approximation to the best feasible solution in hindsight, where $D$ is the sum of delays of gradient feedback. To the best of our knowledge, this is the first result to obtain $O(\sqrt{T})$ regret against a $(1-e^{-\gamma})$-approximation with $O(1)$ gradient inquiry at each time step, when no delay exists, i.e., $D=T$. Finally, numerical experiments demonstrate the effectiveness of our boosting methods.
Noise-injected analog Ising machines enable ultrafast statistical sampling and machine learning
Authors: Fabian Böhm, Diego Alonso-Urquijo, Guy Verschaffelt, Guy Van der Sande
Date: 2021-12-21 00:00:00
Tasks: BIG-bench Machine Learning, Combinatorial Optimization

Ising machines are a promising non-von-Neumann computational concept for neural network training and combinatorial optimization. However, while various neural networks can be implemented with Ising machines, their inability to perform fast statistical sampling makes them inefficient for training neural networks compared to digital computers. Here, we introduce a universal concept to achieve ultrafast statistical sampling with analog Ising machines by injecting noise. With an opto-electronic Ising machine, we experimentally demonstrate that this can be used for accurate sampling of Boltzmann distributions and for unsupervised training of neural networks, with equal accuracy as software-based training. Through simulations, we find that Ising machines can perform statistical sampling orders-of-magnitudes faster than software-based methods. This enables the use of Ising machines beyond combinatorial optimization and makes them into efficient tools for machine learning and other applications.
Neural network training under semidefinite constraints
Authors: Patricia Pauli, Niklas Funcke, Dennis Gramlich, Mohamed Amine Msalmi, Frank Allgöwer
Date: 2022-01-03 00:00:00

This paper is concerned with the training of neural networks (NNs) under semidefinite constraints, which allows for NN training with robustness and stability guarantees. In particular, we focus on Lipschitz bounds for NNs. Exploiting the banded structure of the underlying matrix constraint, we set up an efficient and scalable training scheme for NN training problems of this kind based on interior point methods. Our implementation allows to enforce Lipschitz constraints in the training of large-scale deep NNs such as Wasserstein generative adversarial networks (WGANs) via semidefinite constraints. In numerical examples, we show the superiority of our method and its applicability to WGAN training.
Modelling of Received Signals in Molecular Communication Systems based machine learning: Comparison of azure machine learning and Python tools
Authors: Soha Mohamed, Mahmoud S. Fayed
Date: 2021-12-19 00:00:00
Tasks: BIG-bench Machine Learning, regression

Molecular communication (MC) implemented on Nano networks has extremely attractive characteristics in terms of energy efficiency, dependability, and robustness. Even though, the impact of incredibly slow molecule diffusion and high variability environments remains unknown. Analysis and designs of communication systems usually rely on developing mathematical models that describe the communication channel. However, the underlying channel models are unknown in some systems, such as MC systems, where chemical signals are used to transfer information. In these cases, a new method to analyze and design is needed. In this paper, we concentrate on one critical aspect of the MC system, modelling MC received signal until time t , and demonstrate that using tools from ML makes it promising to train detectors that can be executed well without any information about the channel model. Machine learning (ML) is one of the intelligent methodologies that has shown promising results in the domain. This paper applies Azure Machine Learning (Azure ML) for flexible pavement maintenance regressions problems and solutions. For prediction, four parameters are used as inputs: the receiver radius, transmitter radius, distance between receiver and transmitter, and diffusion coefficient, while the output is mAP (mean average precision) of the received signal. Azure ML enables algorithms that can learn from data and experiences and accomplish tasks without having to be coded. In the established Azure ML, the regression algorithms such as, boost decision tree regression, Bayesian linear regression, neural network, and decision forest regression are selected. The best performance is chosen as an optimality criterion. Finally, a comparison that shows the potential benefits of Azure ML tool over programmed based tool (Python), used by developers on local PCs, is demonstrated
Semi-supervised Stance Detection of Tweets Via Distant Network Supervision
Authors: Subhabrata Dutta, Samiya Caur, Soumen Chakrabarti, Tanmoy Chakraborty
Date: 2022-01-03 00:00:00
Tasks: Hate Speech Detection, Propaganda detection, Stance Detection

Detecting and labeling stance in social media text is strongly motivated by hate speech detection, poll prediction, engagement forecasting, and concerted propaganda detection. Today's best neural stance detectors need large volumes of training data, which is difficult to curate given the fast-changing landscape of social media text and issues on which users opine. Homophily properties over the social network provide strong signal of coarse-grained user-level stance. But semi-supervised approaches for tweet-level stance detection fail to properly leverage homophily. In light of this, We present SANDS, a new semi-supervised stance detector. SANDS starts from very few labeled tweets. It builds multiple deep feature views of tweets. It also uses a distant supervision signal from the social network to provide a surrogate loss signal to the component learners. We prepare two new tweet datasets comprising over 236,000 politically tinted tweets from two demographics (US and India) posted by over 87,000 users, their follower-followee graph, and over 8,000 tweets annotated by linguists. SANDS achieves a macro-F1 score of 0.55 (0.49) on US (India)-based datasets, outperforming 17 baselines (including variants of SANDS) substantially, particularly for minority stance labels and noisy text. Numerous ablation experiments on SANDS disentangle the dynamics of textual and network-propagated stance signals.
Implementation of a Binary Neural Network on a Passive Array of Magnetic Tunnel Junctions
Authors: Jonathan M. Goodwill, Nitin Prasad, Brian D. Hoskins, Matthew W. Daniels, Advait Madhavan, Lei Wan, Tiffany S. Santos, Michael Tran, Jordan A. Katine, Patrick M. Braganca, Mark D. Stiles, Jabez J. McClelland
Date: 2021-12-16 00:00:00

The increasing scale of neural networks and their growing application space have produced demand for more energy- and memory-efficient artificial-intelligence-specific hardware. Avenues to mitigate the main issue, the von Neumann bottleneck, include in-memory and near-memory architectures, as well as algorithmic approaches. Here we leverage the low-power and the inherently binary operation of magnetic tunnel junctions (MTJs) to demonstrate neural network hardware inference based on passive arrays of MTJs. In general, transferring a trained network model to hardware for inference is confronted by degradation in performance due to device-to-device variations, write errors, parasitic resistance, and nonidealities in the substrate. To quantify the effect of these hardware realities, we benchmark 300 unique weight matrix solutions of a 2-layer perceptron to classify the Wine dataset for both classification accuracy and write fidelity. Despite device imperfections, we achieve software-equivalent accuracy of up to 95.3 % with proper tuning of network parameters in 15 x 15 MTJ arrays having a range of device sizes. The success of this tuning process shows that new metrics are needed to characterize the performance and quality of networks reproduced in mixed signal hardware.
'Moving On' -- Investigating Inventors' Ethnic Origins Using Supervised Learning
Authors: Matthias Niggli
Date: 2022-01-03 00:00:00
Tasks: Descriptive

Patent data provides rich information about technical inventions, but does not disclose the ethnic origin of inventors. In this paper, I use supervised learning techniques to infer this information. To do so, I construct a dataset of 95'202 labeled names and train an artificial recurrent neural network with long-short-term memory (LSTM) to predict ethnic origins based on names. The trained network achieves an overall performance of 91% across 17 ethnic origins. I use this model to classify and investigate the ethnic origins of 2.68 million inventors and provide novel descriptive evidence regarding their ethnic origin composition over time and across countries and technological fields. The global ethnic origin composition has become more diverse over the last decades, which was mostly due to a relative increase of Asian origin inventors. Furthermore, the prevalence of foreign-origin inventors is especially high in the USA, but has also increased in other high-income economies. This increase was mainly driven by an inflow of non-western inventors into emerging high-technology fields for the USA, but not for other high-income countries.
Stochastic Weight Averaging Revisited
Authors: Hao Guo, Jiyong Jin, Bin Liu
Date: 2022-01-03 00:00:00

Averaging neural network weights sampled by a backbone stochastic gradient descent (SGD) is a simple yet effective approach to assist the backbone SGD in finding better optima, in terms of generalization. From a statistical perspective, weight averaging (WA) contributes to variance reduction. Recently, a well-established stochastic weight averaging (SWA) method is proposed, which is featured by the application of a cyclical or high constant (CHC) learning rate schedule (LRS) in generating weight samples for WA. Then a new insight on WA appears, which states that WA helps to discover wider optima and then leads to better generalization. We conduct extensive experimental studies for SWA, involving a dozen modern DNN model structures and a dozen benchmark open-source image, graph, and text datasets. We disentangle contributions of the WA operation and the CHC LRS for SWA, showing that the WA operation in SWA still contributes to variance reduction but does not always lead to wide optima. The experimental results indicate that there are global scale geometric structures in the DNN loss landscape. We then present an algorithm termed periodic SWA (PSWA) which makes use of a series of WA operations to discover the global geometric structures. PSWA outperforms its backbone SGD remarkably, providing experimental evidences for the existence of global geometric structures. Codes for reproducing the experimental results are available at https://github.com/ZJLAB-AMMI/PSWA.
Using Non-Stationary Bandits for Learning in Repeated Cournot Games with Non-Stationary Demand
Authors: Kshitija Taywade, Brent Harrison, Judy Goldsmith
Date: 2022-01-03 00:00:00
Tasks: Efficient Exploration

Many past attempts at modeling repeated Cournot games assume that demand is stationary. This does not align with real-world scenarios in which market demands can evolve over a product's lifetime for a myriad of reasons. In this paper, we model repeated Cournot games with non-stationary demand such that firms/agents face separate instances of non-stationary multi-armed bandit problem. The set of arms/actions that an agent can choose from represents discrete production quantities; here, the action space is ordered. Agents are independent and autonomous, and cannot observe anything from the environment; they can only see their own rewards after taking an action, and only work towards maximizing these rewards. We propose a novel algorithm 'Adaptive with Weighted Exploration (AWE) $\epsilon$-greedy' which is remotely based on the well-known $\epsilon$-greedy approach. This algorithm detects and quantifies changes in rewards due to varying market demand and varies learning rate and exploration rate in proportion to the degree of changes in demand, thus enabling agents to better identify new optimal actions. For efficient exploration, it also deploys a mechanism for weighing actions that takes advantage of the ordered action space. We use simulations to study the emergence of various equilibria in the market. In addition, we study the scalability of our approach in terms number of total agents in the system and the size of action space. We consider both symmetric and asymmetric firms in our models. We found that using our proposed method, agents are able to swiftly change their course of action according to the changes in demand, and they also engage in collusive behavior in many simulations.
ELight: Enabling Efficient Photonic In-Memory Neurocomputing with Life Enhancement
Authors: Hanqing Zhu, Jiaqi Gu, Chenghao Feng, Mingjie Liu, Zixuan Jiang, Ray T. Chen, David Z. Pan
Date: 2021-12-15 00:00:00

With the recent advances in optical phase change material (PCM), photonic in-memory neurocomputing has demonstrated its superiority in optical neural network (ONN) designs with near-zero static power consumption, time-of-light latency, and compact footprint. However, photonic tensor cores require massive hardware reuse to implement large matrix multiplication due to the limited single-core scale. The resultant large number of PCM writes leads to serious dynamic power and overwhelms the fragile PCM with limited write endurance. In this work, we propose a synergistic optimization framework, ELight, to minimize the overall write efforts for efficient and reliable optical in-memory neurocomputing. We first propose write-aware training to encourage the similarity among weight blocks, and combine it with a post-training optimization method to reduce programming efforts by eliminating redundant writes. Experiments show that ELight can achieve over 20X reduction in the total number of writes and dynamic power with comparable accuracy. With our ELight, photonic in-memory neurocomputing will step forward towards viable applications in machine learning with preserved accuracy, order-of-magnitude longer lifetime, and lower programming energy.
Transfer-learning-based Surrogate Model for Thermal Conductivity of Nanofluids
Authors: Saeel S. Pai, Abhijeet Banthiya
Date: 2022-01-02 00:00:00
Tasks: Transfer Learning

Heat transfer characteristics of nanofluids have been extensively studied since the 1990s. Research investigations show that the suspended nanoparticles significantly alter the suspension's thermal properties. The thermal conductivity of nanofluids is one of the properties that is generally found to be greater than that of the base fluid. This increase in thermal conductivity is found to depend on several parameters. Several theories have been proposed to model the thermal conductivities of nanofluids, but there is no reliable universal theory yet to model the anomalous thermal conductivity of nanofluids. In recent years, supervised data-driven methods have been successfully employed to create surrogate models across various scientific disciplines, especially for modeling difficult-to-understand phenomena. These supervised learning methods allow the models to capture highly non-linear phenomena. In this work, we have taken advantage of existing correlations and used them concurrently with available experimental results to develop more robust surrogate models for predicting the thermal conductivity of nanofluids. Artificial neural networks are trained using the transfer learning approach to predict the thermal conductivity enhancement of nanofluids with spherical particles for 32 different particle-fluid combinations (8 particles materials and 4 fluids). The large amount of lower accuracy data generated from correlations is used to coarse-tune the model parameters, and the limited amount of more trustworthy experimental data is used to fine-tune the model parameters. The transfer learning-based models' results are compared with those from baseline models which are trained only on experimental data using a goodness of fit metric. It is found that the transfer learning models perform better with goodness of fit values of 0.93 as opposed to 0.83 from the baseline models.
Deep Learning and Linear Programming for Automated Ensemble Forecasting and Interpretation
Authors: Lars Lien Ankile, Kjartan Krange
Date: 2022-01-02 00:00:00
Tasks: Feature Importance, Model Selection, Time Series, Time Series Analysis

This paper presents an ensemble forecasting method that shows strong results on the M4 Competition dataset by decreasing feature and model selection assumptions, termed DONUT (DO Not UTilize human beliefs). Our assumption reductions, primarily consisting of auto-generated features and a more diverse model pool for the ensemble, significantly outperform the statistical, feature-based ensemble method FFORMA by Montero-Manso et al. (2020). We also investigate feature extraction with a Long Short-term Memory Network (LSTM) Autoencoder and find that such features contain crucial information not captured by standard statistical feature approaches. The ensemble weighting model uses LSTM and statistical features to combine the models accurately. The analysis of feature importance and interaction shows a slight superiority for LSTM features over the statistical ones alone. Clustering analysis shows that essential LSTM features differ from most statistical features and each other. We also find that increasing the solution space of the weighting model by augmenting the ensemble with new models is something the weighting model learns to use, thus explaining part of the accuracy gains. Moreover, we present a formal ex-post-facto analysis of an optimal combination and selection for ensembles, quantifying differences through linear optimization on the M4 dataset. Our findings indicate that classical statistical time series features, such as trend and seasonality, alone do not capture all relevant information for forecasting a time series. On the contrary, our novel LSTM features contain significantly more predictive power than the statistical ones alone, but combining the two feature sets proved the best in practice.
Probabilistic Logic Gate in Asynchronous Game of Life with Critical Property
Authors: Yukio-Pegio Gunji, Yoshihiko Ohzawa, Terutaka Tanaka
Date: 2021-12-15 00:00:00

Metaheuristic and self-organizing criticality (SOC) could contribute to robust computation under perturbed environments. Implementing a logic gate in a computing system in a critical state is one of the intriguing ways to study the role of metaheuristics and SOCs. Here, we study the behavior of cellular automaton, game of life (GL), in asynchronous updating and implement probabilistic logic gates by using asynchronous GL. We find that asynchronous GL shows a phase transition, that the density of the state of 1 decays with the power law at the critical point, and that systems at the critical point have the most computability in asynchronous GL. We implement AND and OR gates in asynchronous GL with criticality, which shows good performance. Since tuning perturbations play an essential role in operating logic gates, our study reveals the interference between manipulation and perturbation in probabilistic logic gates.
Succinct Differentiation of Disparate Boosting Ensemble Learning Methods for Prognostication of Polycystic Ovary Syndrome Diagnosis
Authors: Abhishek Gupta, Sannidhi Shetty, Raunak Joshi, Ronald Melwin Laban
Date: 2022-01-02 00:00:00
Tasks: Ensemble Learning

Prognostication of medical problems using the clinical data by leveraging the Machine Learning techniques with stellar precision is one of the most important real world challenges at the present time. Considering the medical problem of Polycystic Ovary Syndrome also known as PCOS is an emerging problem in women aged from 15 to 49. Diagnosing this disorder by using various Boosting Ensemble Methods is something we have presented in this paper. A detailed and compendious differentiation between Adaptive Boost, Gradient Boosting Machine, XGBoost and CatBoost with their respective performance metrics highlighting the hidden anomalies in the data and its effects on the result is something we have presented in this paper. Metrics like Confusion Matrix, Precision, Recall, F1 Score, FPR, RoC Curve and AUC have been used in this paper.
Analog/Mixed-Signal Circuit Synthesis Enabled by the Advancements of Circuit Architectures and Machine Learning Algorithms
Authors: Shiyu Su, Qiaochu Zhang, Mohsen Hassanpourghadi, Juzheng Liu, Rezwan A Rasul, Mike Shuo-Wei Chen
Date: 2021-12-15 00:00:00
Tasks: BIG-bench Machine Learning

Analog mixed-signal (AMS) circuit architecture has evolved towards more digital friendly due to technology scaling and demand for higher flexibility/reconfigurability. Meanwhile, the design complexity and cost of AMS circuits has substantially increased due to the necessity of optimizing the circuit sizing, layout, and verification of a complex AMS circuit. On the other hand, machine learning (ML) algorithms have been under exponential growth over the past decade and actively exploited by the electronic design automation (EDA) community. This paper will identify the opportunities and challenges brought about by this trend and overview several emerging AMS design methodologies that are enabled by the recent evolution of AMS circuit architectures and machine learning algorithms. Specifically, we will focus on using neural-network-based surrogate models to expedite the circuit design parameter search and layout iterations. Lastly, we will demonstrate the rapid synthesis of several AMS circuit examples from specification to silicon prototype, with significantly reduced human intervention.