pwc-archive/papers-with-abstracts huggingface dataset:
Unveiling the role of plasticity rules in reservoir computing
Authors: Guillermo B. Morales, Claudio R. Mirasso, Miguel C. Soriano
Date: 2021-01-14 00:00:00
Tasks: Time Series Analysis, Time Series Prediction

Reservoir Computing (RC) is an appealing approach in Machine Learning that combines the high computational capabilities of Recurrent Neural Networks with a fast and easy training method. Likewise, successful implementation of neuro-inspired plasticity rules into RC artificial networks has boosted the performance of the original models. In this manuscript, we analyze the role that plasticity rules play on the changes that lead to a better performance of RC. To this end, we implement synaptic and non-synaptic plasticity rules in a paradigmatic example of RC model: the Echo State Network. Testing on nonlinear time series prediction tasks, we show evidence that improved performance in all plastic models are linked to a decrease of the pair-wise correlations in the reservoir, as well as a significant increase of individual neurons ability to separate similar inputs in their activity space. Here we provide new insights on this observed improvement through the study of different stages on the plastic learning. From the perspective of the reservoir dynamics, optimal performance is found to occur close to the so-called edge of instability. Our results also show that it is possible to combine different forms of plasticity (namely synaptic and non-synaptic rules) to further improve the performance on prediction tasks, obtaining better results than those achieved with single-plasticity models.
How Shift Equivariance Impacts Metric Learning for Instance Segmentation
Authors: Josef Lorenz Rumberger, Xiaoyan Yu, Peter Hirsch, Melanie Dohmen, Vanessa Emanuela Guarino, Ashkan Mokarian, Lisa Mais, Jan Funke, Dagmar Kainmueller
Date: 2021-01-14 00:00:00
Proceeding: ICCV 2021 10
Tasks: Decoder, Instance Segmentation, Metric Learning, Semantic Segmentation

Metric learning has received conflicting assessments concerning its suitability for solving instance segmentation tasks. It has been dismissed as theoretically flawed due to the shift equivariance of the employed CNNs and their respective inability to distinguish same-looking objects. Yet it has been shown to yield state of the art results for a variety of tasks, and practical issues have mainly been reported in the context of tile-and-stitch approaches, where discontinuities at tile boundaries have been observed. To date, neither of the reported issues have undergone thorough formal analysis. In our work, we contribute a comprehensive formal analysis of the shift equivariance properties of encoder-decoder-style CNNs, which yields a clear picture of what can and cannot be achieved with metric learning in the face of same-looking objects. In particular, we prove that a standard encoder-decoder network that takes $d$-dimensional images as input, with $l$ pooling layers and pooling factor $f$, has the capacity to distinguish at most $f^{dl}$ same-looking objects, and we show that this upper limit can be reached. Furthermore, we show that to avoid discontinuities in a tile-and-stitch approach, assuming standard batch size 1, it is necessary to employ valid convolutions in combination with a training output window size strictly greater than $f^l$, while at test-time it is necessary to crop tiles to size $n\cdot f^l$ before stitching, with $n\geq 1$. We complement these theoretical findings by discussing a number of insightful special cases for which we show empirical results on synthetic data.
Scaling the Convex Barrier with Sparse Dual Algorithms
Authors: Alessandro De Palma, Harkirat Singh Behl, Rudy Bunel, Philip H. S. Torr, M. Pawan Kumar
Date: 2021-01-14 00:00:00
Proceeding: ICLR 2021 1
Tasks: Efficient Neural Network, GPU, valid

Tight and efficient neural network bounding is crucial to the scaling of neural network verification systems. Many efficient bounding algorithms have been presented recently, but they are often too loose to verify more challenging properties. This is due to the weakness of the employed relaxation, which is usually a linear program of size linear in the number of neurons. While a tighter linear relaxation for piecewise-linear activations exists, it comes at the cost of exponentially many constraints and currently lacks an efficient customized solver. We alleviate this deficiency by presenting two novel dual algorithms: one operates a subgradient method on a small active set of dual variables, the other exploits the sparsity of Frank-Wolfe type optimizers to incur only a linear memory cost. Both methods recover the strengths of the new relaxation: tightness and a linear separation oracle. At the same time, they share the benefits of previous dual approaches for weaker relaxations: massive parallelism, GPU implementation, low cost per iteration and valid bounds at any time. As a consequence, we can obtain better bounds than off-the-shelf solvers in only a fraction of their running time, attaining significant formal verification speed-ups.
A Neophyte With AutoML: Evaluating the Promises of Automatic Machine Learning Tools
Authors: Oleg Bezrukavnikov, Rhema Linder
Date: 2021-01-14 00:00:00
Tasks: AutoML, BIG-bench Machine Learning

This paper discusses modern Auto Machine Learning (AutoML) tools from the perspective of a person with little prior experience in Machine Learning (ML). There are many AutoML tools both ready-to-use and under development, which are created to simplify and democratize usage of ML technologies in everyday life. Our position is that ML should be easy to use and available to a greater number of people. Prior research has identified the need for intuitive AutoML tools. This work seeks to understand how well AutoML tools have achieved that goal in practice. We evaluate three AutoML Tools to evaluate the end-user experience and system performance. We evaluate the tools by having them create models from a competition dataset on banking data. We report on their performance and the details of our experience. This process provides a unique understanding of the state of the art of AutoML tools. Finally, we use these experiences to inform a discussion on how future AutoML tools can improve the user experience for neophytes of Machine Learning.
Physics-aware, probabilistic model order reduction with guaranteed stability
Authors: Sebastian Kaltenbach, Phaedon-Stelios Koutsourelakis
Date: 2021-01-14 00:00:00
Proceeding: ICLR 2021 1
Tasks: Dimensionality Reduction, Inductive Bias, Time Series Analysis

Given (small amounts of) time-series' data from a high-dimensional, fine-grained, multiscale dynamical system, we propose a generative framework for learning an effective, lower-dimensional, coarse-grained dynamical model that is predictive of the fine-grained system's long-term evolution but also of its behavior under different initial conditions. We target fine-grained models as they arise in physical applications (e.g. molecular dynamics, agent-based models), the dynamics of which are strongly non-stationary but their transition to equilibrium is governed by unknown slow processes which are largely inaccessible by brute-force simulations. Approaches based on domain knowledge heavily rely on physical insight in identifying temporally slow features and fail to enforce the long-term stability of the learned dynamics. On the other hand, purely statistical frameworks lack interpretability and rely on large amounts of expensive simulation data (long and multiple trajectories) as they cannot infuse domain knowledge. The generative framework proposed achieves the aforementioned desiderata by employing a flexible prior on the complex plane for the latent, slow processes, and an intermediate layer of physics-motivated latent variables that reduces reliance on data and imbues inductive bias. In contrast to existing schemes, it does not require the a priori definition of projection operators from the fine-grained description and addresses simultaneously the tasks of dimensionality reduction and model estimation. We demonstrate its efficacy and accuracy in multiscale physical systems of particle dynamics where probabilistic, long-term predictions of phenomena not contained in the training data are produced.
Needmining: Designing Digital Support to Elicit Needs from Social Media
Authors: Niklas Kühl, Gerhard Satzger
Date: 2021-01-14 00:00:00
Tasks: Management

Today's businesses face a high pressure to innovate in order to succeed in highly competitive markets. Successful innovations, though, typically require the identification and analysis of customer needs. While traditional, established need elicitation methods are time-proven and have demonstrated their capabilities to deliver valuable insights, they lack automation and scalability and, thus, are expensive and time-consuming. In this article, we propose an approach to automatically identify and quantify customer needs by utilizing a novel data source: Users voluntarily and publicly expose information about themselves via social media, as for instance Facebook or Twitter. These posts may contain valuable information about the needs, wants, and demands of their authors. We apply a Design Science Research (DSR) methodology to add design knowledge and artifacts for the digitalization of innovation processes, in particular to provide digital support for the elicitation of customer needs. We want to investigate whether automated, speedy, and scalable need elicitation from social media is feasible. We concentrate on Twitter as a data source and on e-mobility as an application domain. In a first design cycle we conceive, implement and evaluate a method to demonstrate the feasibility of identifying those social media posts that actually express customer needs. In a second cycle, we build on this artifact to additionally quantify the need information elicited, and prove its feasibility. Third, we integrate both developed methods into an end-user software artifact and test usability in an industrial use case. Thus, we add new methods for need elicitation to the body of knowledge, and introduce concrete tooling for innovation management in practice.
Exploration of Visual Features and their weighted-additive fusion for Video Captioning
Authors: Praveen S V, Akhilesh Bharadwaj, Harsh Raj, Janhavi Dadhania, Ganesh Samarth C. A, Nikhil Pareek, S R M Prasanna
Date: 2021-01-14 00:00:00
Tasks: Video Captioning

Video captioning is a popular task that challenges models to describe events in videos using natural language. In this work, we investigate the ability of various visual feature representations derived from state-of-the-art convolutional neural networks to capture high-level semantic context. We introduce the Weighted Additive Fusion Transformer with Memory Augmented Encoders (WAFTM), a captioning model that incorporates memory in a transformer encoder and uses a novel method, to fuse features, that ensures due importance is given to more significant representations. We illustrate a gain in performance realized by applying Word-Piece Tokenization and a popular REINFORCE algorithm. Finally, we benchmark our model on two datasets and obtain a CIDEr of 92.4 on MSVD and a METEOR of 0.091 on the ActivityNet Captions Dataset.
How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
Authors: David Piorkowski, Soya Park, April Yi Wang, Dakuo Wang, Michael Muller, Felix Portnoy
Date: 2021-01-13 00:00:00

The development of AI applications is a multidisciplinary effort, involving multiple roles collaborating with the AI developers, an umbrella term we use to include data scientists and other AI-adjacent roles on the same team. During these collaborations, there is a knowledge mismatch between AI developers, who are skilled in data science, and external stakeholders who are typically not. This difference leads to communication gaps, and the onus falls on AI developers to explain data science concepts to their collaborators. In this paper, we report on a study including analyses of both interviews with AI developers and artifacts they produced for communication. Using the analytic lens of shared mental models, we report on the types of communication gaps that AI developers face, how AI developers communicate across disciplinary and organizational boundaries, and how they simultaneously manage issues regarding trust and expectations.
Knowledge Graphs and Natural-Language Processing
Authors: Andreas L Opdahl
Date: 2020-12-15 00:00:00
Tasks: Knowledge Graphs, Management

Emergency-relevant data comes in many varieties. It can be high volume and high velocity, and reaction times are critical, calling for efficient and powerful techniques for data analysis and management. Knowledge graphs represent data in a rich, flexible, and uniform way that is well matched with the needs of emergency management. They build on existing standards, resources, techniques, and tools for semantic data and computing. This chapter explains the most important semantic technologies and how they support knowledge graphs. We proceed to discuss their benefits and challenges and give examples of relevant semantic data sources and vocabularies. Natural-language texts -- in particular those collected from social media such as Twitter -- is a type of data source that poses particular analysis challenges. We therefore include an overview of techniques for processing natural-language texts.
Approximations with deep neural networks in Sobolev time-space
Authors: Ahmed Abdeljawad, Philipp Grohs
Date: 2020-12-23 00:00:00

Solutions of evolution equation generally lies in certain Bochner-Sobolev spaces, in which the solution may has regularity and integrability properties for the time variable that can be different for the space variables. Therefore, in this paper, we develop a framework shows that deep neural networks can approximate Sobolev-regular functions with respect to Bochner-Sobolev spaces. In our work we use the so-called Rectified Cubic Unit (ReCU) as an activation function in our networks, which allows us to deduce approximation results of the neural networks while avoiding issues caused by the non regularity of the most commonly used Rectivied Linear Unit (ReLU) activation function.
注意！高分辨透射电镜（HRTEM）拍到的钙钛矿可能是卤化铅
Authors: 材料人
Date: 2020-12-29 00:00:00

近日，北京大学已毕业博士Yu-Hao Deng（邓玉豪）在国际期刊《凝聚态》上撰文，全面详细阐述了有机无机杂化钙钛矿高分辨透射电镜（HRTEM）表征中常见的物相与结构的错误标定，并且对这些错误标定进行了重新标定纠正，同时也总结了降低电子束损伤可以得到本征钙钛矿结构的方法，为避免学术界以后再次出现这种错误标定的情况提供了参考。
预警！因忽视了这个细节，数十篇钙钛矿文章的数据恐存在问题
Authors: 纳米人
Date: 2020-12-25 00:00:00

有机无机杂化钙钛矿材料因其优异的光电性能近年来成为研究的热点，但是这类钙钛矿材料存在一个问题就是对电子束辐射特别敏感，在约100个电子每平方埃的电子剂量照射下，有机无机钙钛矿材料就会开始分解为卤化铅，例如甲胺铅碘（MAPbI3）钙钛矿分解为碘化铅（PbI2）。而平时我们进行透射电镜(TEM)进行表征时，电子束的剂量功率却高达数1000个电子每平方埃每秒，在高分辨条件下更是可能高达近万个电子每平方埃每秒，这意味着如果不使用低剂量成像技术，普通的TEM条件几乎不可能得到本征的钙钛矿结构，得到的将是分解后的卤化铅。但是长久以来钙钛矿极易被电子束分解这个问题却被钙钛矿领域的研究人员忽视了，大家在标定钙钛矿物相的时候，如图2所示，很多文章就是测量高分辨TEM照片的晶面间距和夹角进行标定，甚至一些研究人员只测量个晶面间距，夹角都不测量就直接标定材料是钙钛矿。这种标定有一个问题，以MAPbI3为例，在分解产物PbI2中，会存在和钙钛矿材料晶面间距以及夹角都极其相似的一组晶面，那么这种不严谨的标定方法就会导致将分解后的PbI2标定为MAPbI3，这种错误的标定现在普遍存在于电镜表征钙钛矿材料的文献中。标定高分辨TEM数据，只比较一组晶面间距和晶面夹角的标定方法是不严谨的。正确的标定除了晶面间距和晶面夹角要与目标材料一致，另外还必须要保证存在的晶面是完整的，如果是电子衍射（ED）那就是衍射斑点需要是完整的，这就需要同时将高分辨TEM照片、电子衍射数据与衍射图样、XRD数据或者模拟的衍射图进行比较，在确定晶面间距匹配，晶面夹角匹配，并且晶面没有缺失这三个前提条件下，才可以说你表征得到的材料是钙钛矿，而不是其他分解中间体。
Pulmonary drug delivery and retention: a computational study to identify plausible parameters based on a coupled airway-mucus flow model
Authors: Aranyak Chakravarty, Mahesh V. Panchagnula, Alladi Mohan, Neelesh A. Patankar
Date: 2020-11-21 00:00:00

Pulmonary drug delivery systems rely on inhalation of drug-laden aerosols produced from aerosol generators such as inhalers, nebulizers etc. On deposition, the drug molecules diffuse in the mucus layer and are also subjected to mucociliary advection which transports the drugs away from the initial deposition site. The availability of the drug at a particular region of the lung is, thus, determined by a balance between these two phenomena. A mathematical analysis of drug deposition and retention in the lungs is developed through a coupled mathematical model of aerosol transport in air as well as drug molecule transport in the mucus layer. The mathematical model is solved computationally to identify suitable conditions for the transport of drug-laden aerosols to the deep lungs. This study identifies the conditions conducive for delivering drugs to the deep lungs which is crucial for achieving systemic drug delivery. The effect of different parameters on drug retention is also characterized for various regions of the lungs, which is important in determining the availability of the inhaled drugs at a target location. Our analysis confirms that drug delivery efficacy remains highest for aerosols in the size range of 1-5 $\mu$m. Moreover, it is observed that amount of drugs deposited in the deep lung increases by a factor of 2 when the breathing time period is doubled, with respect to normal breathing, suggesting breath control as a means to increase the efficacy of drug delivery to the deep lung. A higher efficacy also reduces the drug load required to be inhaled to produce the same health effects and hence, can help in minimizing the side effects of a drug.
Modelling Universal Order Book Dynamics in Bitcoin Market
Authors: Fabin Shi, Nathan Aden, Shengda Huang, Neil Johnson, Xiaoqian Sun, Jinhua Gao, Li Xu, HuaWei Shen, Xueqi Cheng, Chaoming Song
Date: 2021-01-15 00:00:00

Understanding the emergence of universal features such as the stylized facts in markets is a long-standing challenge that has drawn much attention from economists and physicists. Most existing models, such as stochastic volatility models, focus mainly on price changes, neglecting the complex trading dynamics. Recently, there are increasing studies on order books, thanks to the availability of large-scale trading datasets, aiming to understand the underlying mechanisms governing the market dynamics. In this paper, we collect order-book datasets of Bitcoin platforms across three countries over millions of users and billions of daily turnovers. We find a 1+1D field theory, govern by a set of KPZ-like stochastic equations, predicts precisely the order book dynamics observed in empirical data. Despite the microscopic difference of markets, we argue the proposed effective field theory captures the correct universality class of market dynamics. We also show that the model agrees with the existing stochastic volatility models at the long-wavelength limit.
Adequacy of time-series reduction for renewable energy systems
Authors: Leonard Göke, Mario Kendziorski
Date: 2021-01-15 00:00:00
Tasks: Time Series, Time Series Analysis

To reduce computational complexity, macro-energy system models commonly implement reduced time-series data. For renewable energy systems dependent on seasonal storage and characterized by intermittent renewables, like wind and solar, adequacy of time-series reduction is in question. Using a capacity expansion model, we evaluate different methods for creating and implementing reduced time-series regarding loss of load and system costs. Results show that adequacy greatly depends on the length of the reduced time-series and how it is implemented into the model. Implementation as a chronological sequence with re-scaled time-steps prevents loss of load best but imposes a positive bias on seasonal storage resulting in an overestimation of system costs. Compared to chronological sequences, grouped periods require more time so solve for the same number of time-steps, because the approach requires additional variables and constraints. Overall, results suggest further efforts to improve time-series reduction and other methods for reducing computational complexity.
A Hitchhiker's Guide to Structural Similarity
Authors: Abhinau K. Venkataramanan, Chengyang Wu, Alan C. Bovik, Ioannis Katsavounidis, Zafar Shahid
Date: 2021-01-16 00:00:00
Tasks: MS-SSIM, SSIM

The Structural Similarity (SSIM) Index is a very widely used image/video quality model that continues to play an important role in the perceptual evaluation of compression algorithms, encoding recipes and numerous other image/video processing algorithms. Several public implementations of the SSIM and Multiscale-SSIM (MS-SSIM) algorithms have been developed, which differ in efficiency and performance. This "bendable ruler" makes the process of quality assessment of encoding algorithms unreliable. To address this situation, we studied and compared the functions and performances of popular and widely used implementations of SSIM, and we also considered a variety of design choices. Based on our studies and experiments, we have arrived at a collection of recommendations on how to use SSIM most effectively, including ways to reduce its computational burden.
AMFFCN: Attentional Multi-layer Feature Fusion Convolution Network for Audio-visual Speech Enhancement
Authors: Xinmeng Xu, Jianjun Hao
Date: 2021-01-15 00:00:00
Tasks: Decoder, Speech Enhancement

Audio-visual speech enhancement system is regarded to be one of promising solutions for isolating and enhancing speech of desired speaker. Conventional methods focus on predicting clean speech spectrum via a naive convolution neural network based encoder-decoder architecture, and these methods a) not adequate to use data fully and effectively, b) cannot process features selectively. The proposed model addresses these drawbacks, by a) applying a model that fuses audio and visual features layer by layer in encoding phase, and that feeds fused audio-visual features to each corresponding decoder layer, and more importantly, b) introducing soft threshold attention into the model to select the informative modality softly. This paper proposes attentional audio-visual multi-layer feature fusion model, in which soft threshold attention unit are applied on feature mapping at every layer of decoder. The proposed model demonstrates the superior performance of the network against the state-of-the-art models.
Bayesian Inference Forgetting
Authors: Shaopeng Fu, Fengxiang He, Yue Xu, DaCheng Tao
Date: 2021-01-16 00:00:00
Tasks: Bayesian Inference, Variational Inference

The right to be forgotten has been legislated in many countries but the enforcement in machine learning would cause unbearable costs: companies may need to delete whole models learned from massive resources due to single individual requests. Existing works propose to remove the knowledge learned from the requested data via its influence function which is no longer naturally well-defined in Bayesian inference. This paper proposes a {\it Bayesian inference forgetting} (BIF) framework to realize the right to be forgotten in Bayesian inference. In the BIF framework, we develop forgetting algorithms for variational inference and Markov chain Monte Carlo. We show that our algorithms can provably remove the influence of single datums on the learned models. Theoretical analysis demonstrates that our algorithms have guaranteed generalizability. Experiments of Gaussian mixture models on the synthetic dataset and Bayesian neural networks on the real-world data verify the feasibility of our methods. The source code package is available at \url{https://github.com/fshp971/BIF}.