pwc-archive/papers-with-abstracts huggingface dataset:
Channel Attention Is All You Need for Video Frame Interpolation
Authors: Myungsub Choi, Heewon Kim, Bohyung Han, Ning Xu, Kyoung Mu Lee
Date: 2020-04-03 00:00:00
Proceeding: AAAI Conference on Artificial  Intelligence 2020 4
Tasks: All, Motion Estimation, Optical Flow Estimation, Video Frame Interpolation

Prevailing video frame interpolation techniques rely heavily on optical flow estimation and require additional model complexity and computational cost; it is also susceptible to error propagation in challenging scenarios with large motion and heavy occlusion. To alleviate the limitation, we propose a simple but effective deep neural network for video frame interpolation, which is end-to-end trainable and is free from a motion estimation network component. Our algorithm employs a special feature reshaping operation, referred to as PixelShuffle, with a channel attention, which replaces the optical flow computation module. The main idea behind the design is to distribute the information in a feature map into multiple channels and extract motion information by attending the channels for pixel-level frame synthesis. The model given by this principle turns out to be effective in the presence of challenging motion and occlusion. We construct a comprehensive evaluation benchmark and demonstrate that the proposed approach achieves outstanding performance compared to the existing models with a component for optical flow computation.
Throughput Optimization for Grant-Free Multiple Access With Multiagent Deep Reinforcement Learning
Authors: Rui Huang, Vincent W.S. Wong, Robert Schober
Date: 2021-02-01 00:00:00
Proceeding: IEEE Transactions on Wireless Communications 2021 2
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

Grant-free multiple access (GFMA) is a promising paradigm to efficiently support uplink access of Internet of Things (IoT) devices. In this paper, we propose a deep reinforcement learning (DRL)-based pilot sequence selection scheme for GFMA systems to mitigate potential pilot sequence collisions. We formulate a pilot sequence selection problem for aggregate throughput maximization in GFMA systems with specific throughput constraints as a Markov decision process (MDP). By exploiting multiagent DRL, we train deep neural networks (DNNs) to learn near-optimal pilot sequence selection policies from the transition history of the underlying MDP without requiring information exchange between the users. While the training process takes advantage of global information, we leverage the technique of factorization to ensure that the policies learned by the DNNs can be executed in a distributed manner. Simulation results show that the proposed scheme can achieve an average aggregate throughput that is within 85% of the optimum, and is 31%, 128%, and 162% higher than that of acknowledgement-based GFMA, dynamic access class barring, and random selection GFMA, respectively. Our results also demonstrate the capability of the proposed scheme to support IoT devices with specific throughput requirements.
Multi-Paradigm Analysis of Thai Capital Market Linkages: Bivariate/Vine Copulas, Granger Causality, Network Centrality, and Graph Neural Network/Graph Embedding Approaches
Authors: Poomjai Nacaskul, Kongkan Kalakan, Pitikorn Khlaisamniang, Puvarith Veerabulyarith, Isariyaporn Sukcharoenchaikul
Date: 2023-02-23 00:00:00
Proceeding: www.researchgate.net 2023 2
Tasks: Econometrics, Graph Embedding, Graph Neural Network

Analytically thorough understanding of causal, probabilistic, and informational linkages amongst modern, highly-interconnected capital markets is fundamental to the promotion of capital-market innovation, efficiency, and resilience; whereupon innovative, efficient, and resilient capital markets are fundamental to the sustainable economic development of any nation and the robust financial stability of her economy. Whereas analytics derived essentially from some form of linear correlation matrix remains to this day the archetype model representation of return co-movements within a single stock exchange, analysis of financial random multi-variates, especially across domestic/international/regional/global/ domains, warrants a generalised multi-paradigm investigation. In lieu of employing a particular quantitative technique to test a specific theoretically-motivated hypothesis, our aim here is to apply a multitude of analytical tools from semi-/non-parametric statistics, from time-series econometrics, from graph-theoretic network analysis, and from Graph Neural Network (GNN)/Geometric Deep Learning (GDL) paradigms to investigate quantifiable linkages amongst 14 selected variables representing Thai/Asia/Emerging Market/US foreign-exchange, fixed-income, and equity market movements, as well as foreign portfolio-investment flows into Thailand’s domestic equity and bond markets. In particular, Bivariate/Vine Copulas, Granger Causality, Network Centrality, and GNN/Graph Embedding together enable us to tease out subtle quantitative relations amongst said linked capital-market variables, which could in turn be exploited, for instance, in the form of better specified Monte Carlo Value-at-Risk/Expected Shortfall estimation for hypothetical portfolios comprising multi-national assets. Our experiments thus far delivered incremental, rather than game-changing, improvements w.r.t. market insights and accuracy gains. Nonetheless, our multi-paradigm framework, along with the publicly shared codebase, can be applied to any other “linkage” phenomena of future research interest and real-world application.
Ecological Sampling of Gaze Shifts
Authors: Giuseppe Boccignone, Mario Ferraro
Date: 2013-04-16 00:00:00
Proceeding: IEEE Transactions on Cybernetics 2013 4
Tasks: Gaze Estimation, Gaze Prediction

Visual attention guides our gaze to relevant parts of the viewed scene, yet the moment-to-moment relocation of gaze can be different among observers even though the same locations are taken into account. Surprisingly, the variability of eye movements has been so far overlooked by the great majority of computational models of visual attention. In this paper we present the ecological sampling model, a stochastic model of eye guidance explaining such variability. The gaze shift mechanism is conceived as an active random sampling that the foraging eye carries out upon the visual landscape, under the constraints set by the observable features and the global complexity of the landscape. By drawing on results reported in the foraging literature, the actual gaze relocation is eventually driven by a stochastic differential equation whose noise source is sampled from a mixture of α-stable distributions. This way, the sampling strategy proposed here allows to mimic a fundamental property of the eye guidance mechanism: where we choose to look next at any given moment in time, it is not completely deterministic, but neither is it completely random To show that the model yields gaze shift motor behaviors that exhibit statistics similar to those displayed by human observers, we compare simulation outputs with those obtained from eye-tracked subjects while viewing complex dynamic scenes.
HLSDataset: Open-Source Dataset for ML-Assisted FPGA Design using High Level Synthesis
Authors: Zhigang Wei, Aman Arora, Ruihao Li, Lizy K. John
Date: 2023-02-17 00:00:00
Tasks: High-Level Synthesis

Machine Learning (ML) has been widely adopted in design exploration using high level synthesis (HLS) to give a better and faster performance, and resource and power estimation at very early stages for FPGA-based design. To perform prediction accurately, high-quality and large-volume datasets are required for training ML models.This paper presents a dataset for ML-assisted FPGA design using HLS, called HLSDataset. The dataset is generated from widely used HLS C benchmarks including Polybench, Machsuite, CHStone and Rossetta. The Verilog samples are generated with a variety of directives including loop unroll, loop pipeline and array partition to make sure optimized and realistic designs are covered. The total number of generated Verilog samples is nearly 9,000 per FPGA type. To demonstrate the effectiveness of our dataset, we undertake case studies to perform power estimation and resource usage estimation with ML models trained with our dataset. All the codes and dataset are public at the github repo.We believe that HLSDataset can save valuable time for researchers by avoiding the tedious process of running tools, scripting and parsing files to generate the dataset, and enable them to spend more time where it counts, that is, in training ML models.
Deep Neural Networks for Encrypted Inference with TFHE
Authors: Andrei Stoian, Jordan Frery, Roman Bredehoft, Luis Montero, Celia Kherfallah, Benoit Chevallier-Mames
Date: 2023-02-13 00:00:00
Tasks: Privacy Preserving

Fully homomorphic encryption (FHE) is an encryption method that allows to perform computation on encrypted data, without decryption. FHE preserves the privacy of the users of online services that handle sensitive data, such as health data, biometrics, credit scores and other personal information. A common way to provide a valuable service on such data is through machine learning and, at this time, Neural Networks are the dominant machine learning model for unstructured data. In this work we show how to construct Deep Neural Networks (DNN) that are compatible with the constraints of TFHE, an FHE scheme that allows arbitrary depth computation circuits. We discuss the constraints and show the architecture of DNNs for two computer vision tasks. We benchmark the architectures using the Concrete stack, an open-source implementation of TFHE.
Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment
Authors: Alejandro Peña, Ignacio Serna, Aythami Morales, Julian Fierrez, Alfonso Ortega, Ainhoa Herrarte, Manuel Alcantara, Javier Ortega-Garcia
Date: 2023-02-13 00:00:00
Tasks: Decision Making, Fairness

The presence of decision-making algorithms in society is rapidly increasing nowadays, while concerns about their transparency and the possibility of these algorithms becoming new sources of discrimination are arising. There is a certain consensus about the need to develop AI applications with a Human-Centric approach. Human-Centric Machine Learning needs to be developed based on four main requirements: (i) utility and social good; (ii) privacy and data ownership; (iii) transparency and accountability; and (iv) fairness in AI-driven decision-making processes. All these four Human-Centric requirements are closely related to each other. With the aim of studying how current multimodal algorithms based on heterogeneous sources of information are affected by sensitive elements and inner biases in the data, we propose a fictitious case study focused on automated recruitment: FairCVtest. We train automatic recruitment algorithms using a set of multimodal synthetic profiles including image, text, and structured data, which are consciously scored with gender and racial biases. FairCVtest shows the capacity of the Artificial Intelligence (AI) behind automatic recruitment tools built this way (a common practice in many other application scenarios beyond recruitment) to extract sensitive information from unstructured data and exploit it in combination to data biases in undesirable (unfair) ways. We present an overview of recent works developing techniques capable of removing sensitive information and biases from the decision-making process of deep learning architectures, as well as commonly used databases for fairness research in AI. We demonstrate how learning approaches developed to guarantee privacy in latent spaces can lead to unbiased and fair automatic decision-making process.
Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness
Authors: Felix Friedrich, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni, Kristian Kersting
Date: 2023-02-07 00:00:00
Tasks: Fairness, Image Generation, Text to Image Generation, Text-to-Image Generation

Generative AI models have recently achieved astonishing results in quality and are consequently employed in a fast-growing number of applications. However, since they are highly data-driven, relying on billion-sized datasets randomly scraped from the internet, they also suffer from degenerated and biased human behavior, as we demonstrate. In fact, they may even reinforce such biases. To not only uncover but also combat these undesired effects, we present a novel strategy, called Fair Diffusion, to attenuate biases after the deployment of generative text-to-image models. Specifically, we demonstrate shifting a bias, based on human instructions, in any direction yielding arbitrarily new proportions for, e.g., identity groups. As our empirical evaluation demonstrates, this introduced control enables instructing generative image models on fairness, with no data filtering and additional training required.
Towards a responsible machine learning approach to identify forced labor in fisheries
Authors: Rocío Joo, Gavin McDonald, Nathan Miller, David Kroodsma, Courtney Farthing, Dyhia Belhabib, Timothy Hochberg
Date: 2023-02-03 00:00:00

Many fishing vessels use forced labor, but identifying vessels that engage in this practice is challenging because few are regularly inspected. We developed a positive-unlabeled learning algorithm using vessel characteristics and movement patterns to estimate an upper bound of the number of positive cases of forced labor, with the goal of helping make accurate, responsible, and fair decisions. 89% of the reported cases of forced labor were correctly classified as positive (recall) while 98% of the vessels certified as having decent working conditions were correctly classified as negative. The recall was high for vessels from different regions using different gears, except for trawlers. We found that as much as ~28% of vessels may operate using forced labor, with the fraction much higher in squid jiggers and longlines. This model could inform risk-based port inspections as part of a broader monitoring, control, and surveillance regime to reduce forced labor. * Translated versions of the English title and abstract are available in five languages in S1 Text: Spanish, French, Simplified Chinese, Traditional Chinese, and Indonesian.
Scientific Computing with Diffractive Optical Neural Networks
Authors: Ruiyang Chen, Yingheng Tang, Jianzhu Ma, Weilu Gao
Date: 2023-02-12 00:00:00
Tasks: Feature Engineering, image-classification, Image Classification

Diffractive optical neural networks (DONNs) have been emerging as a high-throughput and energy-efficient hardware platform to perform all-optical machine learning (ML) in machine vision systems. However, the current demonstrated applications of DONNs are largely straightforward image classification tasks, which undermines the prospect of developing and utilizing such hardware for other ML applications. Here, we numerically and experimentally demonstrate the deployment of an all-optical reconfigurable DONNs system for scientific computing, including guiding two-dimensional quantum material synthesis, predicting the properties of nanomaterials and small molecular cancer drugs, predicting the device response of nanopatterned integrated photonic power splitters, and the dynamic stabilization of an inverted pendulum with reinforcement learning. Despite a large variety of input data structures, we develop a universal feature engineering approach to convert categorical input features to the images that can be processed in the DONNs system. Our results open up new opportunities of employing DONNs systems for a broad range of ML applications.
Nonasymptotic Performance Analysis of Direct-Augmentation and Spatial-Smoothing ESPRIT for Localization of More Sources Than Sensors Using Sparse Arrays
Authors: Zai Yang, Kaijie Wang
Date: 2023-02-22 00:00:00

Direction augmentation (DA) and spatial smoothing (SS), followed by a subspace method such as ESPRIT or MUSIC, are two simple and successful approaches that enable localization of more uncorrelated sources than sensors with a proper sparse array. In this paper, we carry out nonasymptotic performance analyses of DA-ESPRIT and SS-ESPRIT in the practical finite-snapshot regime. We show that their absolute localization errors are bounded from above by $C_1\frac{\max\{\sigma^2, C_2\}}{\sqrt{L}}$ with overwhelming probability, where $L$ is the snapshot number, $\sigma^2$ is the Gaussian noise power, and $C_1,C_2$ are constants independent of $L$ and $\sigma^2$, if and only if they can do exact source localization with infinitely many snapshots. We also show that their resolution increases with the snapshot number, without a substantial limit. Numerical results corroborating our analysis are provided.
Feature Affinity Assisted Knowledge Distillation and Quantization of Deep Neural Networks on Label-Free Data
Authors: Zhijian Li, Biao Yang, Penghang Yin, Yingyong Qi, Jack Xin
Date: 2023-02-10 00:00:00
Tasks: Knowledge Distillation, Quantization

In this paper, we propose a feature affinity (FA) assisted knowledge distillation (KD) method to improve quantization-aware training of deep neural networks (DNN). The FA loss on intermediate feature maps of DNNs plays the role of teaching middle steps of a solution to a student instead of only giving final answers in the conventional KD where the loss acts on the network logits at the output level. Combining logit loss and FA loss, we found that the quantized student network receives stronger supervision than from the labeled ground-truth data. The resulting FAQD is capable of compressing model on label-free data, which brings immediate practical benefits as pre-trained teacher models are readily available and unlabeled data are abundant. In contrast, data labeling is often laborious and expensive. Finally, we propose a fast feature affinity (FFA) loss that accurately approximates FA loss with a lower order of computational complexity, which helps speed up training for high resolution image input.
Balanced Audiovisual Dataset for Imbalance Analysis
Authors: Wenke Xia, Xu Zhao, Xincheng Pang, Changqing Zhang, Di Hu
Date: 2023-02-14 00:00:00

The imbalance problem is widespread in the field of machine learning, which also exists in multimodal learning areas caused by the intrinsic discrepancy between modalities of samples. Recent works have attempted to solve the modality imbalance problem from algorithm perspective, however, they do not fully analyze the influence of modality bias in datasets. Concretely, existing multimodal datasets are usually collected under specific tasks, where one modality tends to perform better than other ones in most conditions. In this work, to comprehensively explore the influence of modality bias, we first split existing datasets into different subsets by estimating sample-wise modality discrepancy. We surprisingly find that: the multimodal models with existing imbalance algorithms consistently perform worse than the unimodal one on specific subsets, in accordance with the modality bias. To further explore the influence of modality bias and analyze the effectiveness of existing imbalance algorithms, we build a balanced audiovisual dataset, with uniformly distributed modality discrepancy over the whole dataset. We then conduct extensive experiments to re-evaluate existing imbalance algorithms and draw some interesting findings: existing algorithms only provide a compromise between modalities and suffer from the large modality discrepancy of samples. We hope that these findings could facilitate future research on the modality imbalance problem.
Visual Watermark Removal Based on Deep Learning
Authors: Rongfeng Wei
Date: 2023-02-07 00:00:00
Tasks: Deep Learning

In recent years as the internet age continues to grow, sharing images on social media has become a common occurrence. In certain cases, watermarks are used as protection for the ownership of the image, however, in more cases, one may wish to remove these watermark images to get the original image without obscuring. In this work, we proposed a deep learning method based technique for visual watermark removal. Inspired by the strong image translation performance of the U-structure, an end-to-end deep neural network model named AdvancedUnet is proposed to extract and remove the visual watermark simultaneously. On the other hand, we embed some effective RSU module instead of the common residual block used in UNet, which increases the depth of the whole architecture without significantly increasing the computational cost. The deep-supervised hybrid loss guides the network to learn the transformation between the input image and the ground truth in a multi-scale and three-level hierarchy. Comparison experiments demonstrate the effectiveness of our method.
Contrastive Representation Learning for Acoustic Parameter Estimation
Authors: Philipp Götz, Cagdas Tuna, Andreas Walther, Emanuël A. P. Habets
Date: 2023-02-22 00:00:00
Tasks: Contrastive Learning, Data Augmentation, parameter estimation, regression, Representation Learning

A study is presented in which a contrastive learning approach is used to extract low-dimensional representations of the acoustic environment from single-channel, reverberant speech signals. Convolution of room impulse responses (RIRs) with anechoic source signals is leveraged as a data augmentation technique that offers considerable flexibility in the design of the upstream task. We evaluate the embeddings across three different downstream tasks, which include the regression of acoustic parameters reverberation time RT60 and clarity index C50, and the classification into small and large rooms. We demonstrate that the learned representations generalize well to unseen data and perform similarly to a fully-supervised baseline.
Do Orcas Have Semantic Language? Machine Learning to Predict Orca Behaviors Using Partially Labeled Vocalization Data
Authors: Sophia Sandholm
Date: 2023-01-28 00:00:00

Orcinus orca (killer whales) exhibit complex calls. They last about a second. In a call, an orca typically uses multiple frequencies simultaneously, varies the frequencies, and varies their volumes. Behavior data is hard to obtain because orcas live under water and travel quickly. Sound data is relatively easy to capture. As a science goal, we would like to know whether orca vocalizations constitute a semantic language. We do this by studying whether machine learning can predict behavior from vocalizations. Such prediction would also help scientific research and safety applications because one would like to predict behavior while only having to capture sound. A significant challenge in this process is lack of labeled data. We work with recent recordings of McMurdo Sound orcas [Wellard et al. 2020] where each recording is labeled with the behaviors observed during the recording. This yields a dataset where sound segments - continuous vocalizations that can be thought of as call sequences or more general structures - within the recordings are labeled with superfluous behaviors. Despite that, with a careful combination of recent machine learning techniques, we achieve 96.4% classification accuracy. This suggests that orcas do use a semantic language. It is also promising for research and applications.