pwc-archive/papers-with-abstracts huggingface dataset:
Mining Effective Features Using Quantum Entropy for Humor Recognition
Authors: Yang Liu, Yuexian Hou
Date: 2023-02-07 00:00:00

Humor recognition has been extensively studied with different methods in the past years. However, existing studies on humor recognition do not understand the mechanisms that generate humor. In this paper, inspired by the incongruity theory, any joke can be divided into two components (the setup and the punchline). Both components have multiple possible semantics, and there is an incongruous relationship between them. We use density matrices to represent the semantic uncertainty of the setup and the punchline, respectively, and design QE-Uncertainty and QE-Incongruity with the help of quantum entropy as features for humor recognition. The experimental results on the SemEval2021 Task 7 dataset show that the proposed features are more effective than the baselines for recognizing humorous and non-humorous texts.
CRAFT: Criticality-Aware Fault-Tolerance Enhancement Techniques for Emerging Memories-Based Deep Neural Networks
Authors: Thai-Hoang Nguyen, Muhammad Imran, Jaehyuk Choi, Joon-Sung Yang
Date: 2023-02-08 00:00:00

Deep Neural Networks (DNNs) have emerged as the most effective programming paradigm for computer vision and natural language processing applications. With the rapid development of DNNs, efficient hardware architectures for deploying DNN-based applications on edge devices have been extensively studied. Emerging Non-Volatile Memories (NVMs), with their better scalability, non-volatility and good read performance, are found to be promising candidates for deploying DNNs. However, despite the promise, emerging NVMs often suffer from reliability issues such as stuck-at faults, which decrease the chip yield/memory lifetime and severely impact the accuracy of DNNs. A stuck-at cell can be read but not reprogrammed, thus, stuck-at faults in NVMs may or may not result in errors depending on the data to be stored. By reducing the number of errors caused by stuck-at faults, the reliability of a DNN-based system can be enhanced. This paper proposes CRAFT, i.e., Criticality-Aware Fault-Tolerance Enhancement Techniques to enhance the reliability of NVM-based DNNs in the presence of stuck-at faults. A data block remapping technique is used to reduce the impact of stuck-at faults on DNNs accuracy. Additionally, by performing bit-level criticality analysis on various DNNs, the critical-bit positions in network parameters that can significantly impact the accuracy are identified. Based on this analysis, we propose an encoding method which effectively swaps the critical bit positions with that of non-critical bits when more errors (due to stuck-at faults) are present in the critical bits.
Inverse Models for Estimating the Initial Condition of Spatio-Temporal Advection-Diffusion Processes
Authors: Xiao Liu, Kyongmin Yeo
Date: 2023-02-08 00:00:00

Inverse problems involve making inference about unknown parameters of a physical process using observational data. This paper investigates an important class of inverse problems -- the estimation of the initial condition of a spatio-temporal advection-diffusion process using spatially sparse data streams. Three spatial sampling schemes are considered, including irregular, non-uniform and shifted uniform sampling. The irregular sampling scheme is the general scenario, while computationally efficient solutions are available in the spectral domain for non-uniform and shifted uniform sampling. For each sampling scheme, the inverse problem is formulated as a regularized convex optimization problem that minimizes the distance between forward model outputs and observations. The optimization problem is solved by the Alternating Direction Method of Multipliers algorithm, which also handles the situation when a linear inequality constraint (e.g., non-negativity) is imposed on the model output. Numerical examples are presented, code is made available on GitHub, and discussions are provided to generate some useful insights of the proposed inverse modeling approaches.
Extragradient-Type Methods with $\mathcal{O} (1/k)$ Last-Iterate Convergence Rates for Co-Hypomonotone Inclusions
Authors: Quoc Tran-Dinh
Date: 2023-02-08 00:00:00

We develop two "Nesterov's accelerated" variants of the well-known extragradient method to approximate a solution of a co-hypomonotone inclusion constituted by the sum of two operators, where one is Lipschitz continuous and the other is possibly multivalued. The first scheme can be viewed as an accelerated variant of Tseng's forward-backward-forward splitting (FBFS) method, while the second one is a Nesterov's accelerated variant of the "past" FBFS scheme, which requires only one evaluation of the Lipschitz operator and one resolvent of the multivalued mapping. Under appropriate conditions on the parameters, we theoretically prove that both algorithms achieve $\mathcal{O}(1/k)$ last-iterate convergence rates on the residual norm, where $k$ is the iteration counter. Our results can be viewed as alternatives of a recent class of Halpern-type methods for root-finding problems. For comparison, we also provide a new convergence analysis of the two recent extra-anchored gradient-type methods for solving co-hypomonotone inclusions.
Gibbsian polar slice sampling
Authors: Philip Schär, Michael Habeck, Daniel Rudolf
Date: 2023-02-08 00:00:00

Polar slice sampling (Roberts & Rosenthal, 2002) is a Markov chain approach for approximate sampling of distributions that is difficult, if not impossible, to implement efficiently, but behaves provably well with respect to the dimension. By updating the directional and radial components of chain iterates separately, we obtain a family of samplers that mimic polar slice sampling, and yet can be implemented efficiently. Numerical experiments in a variety of settings indicate that our proposed algorithm outperforms the two most closely related approaches, elliptical slice sampling (Murray et al., 2010) and hit-and-run uniform slice sampling (MacKay, 2003). We prove the well-definedness and convergence of our methods under suitable assumptions on the target distribution.
An Empirical Study of Uniform-Architecture Knowledge Distillation in Document Ranking
Authors: Xubo Qin, Xiyuan Liu, Xiongfeng Zheng, Jie Liu, Yutao Zhu
Date: 2023-02-08 00:00:00
Tasks: Document Ranking, Knowledge Distillation

Although BERT-based ranking models have been commonly used in commercial search engines, they are usually time-consuming for online ranking tasks. Knowledge distillation, which aims at learning a smaller model with comparable performance to a larger model, is a common strategy for reducing the online inference latency. In this paper, we investigate the effect of different loss functions for uniform-architecture distillation of BERT-based ranking models. Here "uniform-architecture" denotes that both teacher and student models are in cross-encoder architecture, while the student models include small-scaled pre-trained language models. Our experimental results reveal that the optimal distillation configuration for ranking tasks is much different than general natural language processing tasks. Specifically, when the student models are in cross-encoder architecture, a pairwise loss of hard labels is critical for training student models, whereas the distillation objectives of intermediate Transformer layers may hurt performance. These findings emphasize the necessity of carefully designing a distillation strategy (for cross-encoder student models) tailored for document ranking with pairwise training samples.
Dual-interest Factorization-heads Attention for Sequential Recommendation
Authors: GuanYu Lin, Chen Gao, Yu Zheng, Jianxin Chang, Yanan Niu, Yang song, Zhiheng Li, Depeng Jin, Yong Li
Date: 2023-02-08 00:00:00
Tasks: Disentanglement, Sequential Recommendation

Accurate user interest modeling is vital for recommendation scenarios. One of the effective solutions is the sequential recommendation that relies on click behaviors, but this is not elegant in the video feed recommendation where users are passive in receiving the streaming contents and return skip or no-skip behaviors. Here skip and no-skip behaviors can be treated as negative and positive feedback, respectively. With the mixture of positive and negative feedback, it is challenging to capture the transition pattern of behavioral sequence. To do so, FeedRec has exploited a shared vanilla Transformer, which may be inelegant because head interaction of multi-heads attention does not consider different types of feedback. In this paper, we propose Dual-interest Factorization-heads Attention for Sequential Recommendation (short for DFAR) consisting of feedback-aware encoding layer, dual-interest disentangling layer and prediction layer. In the feedback-aware encoding layer, we first suppose each head of multi-heads attention can capture specific feedback relations. Then we further propose factorization-heads attention which can mask specific head interaction and inject feedback information so as to factorize the relation between different types of feedback. Additionally, we propose a dual-interest disentangling layer to decouple positive and negative interests before performing disentanglement on their representations. Finally, we evolve the positive and negative interests by corresponding towers whose outputs are contrastive by BPR loss. Experiments on two real-world datasets show the superiority of our proposed method against state-of-the-art baselines. Further ablation study and visualization also sustain its effectiveness. We release the source code here: https://github.com/tsinghua-fib-lab/WWW2023-DFAR.
Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems
Authors: Peng Liu, Lemei Zhang, Jon Atle Gulla
Date: 2023-02-07 00:00:00
Tasks: Language Modelling, Recommendation Systems

The emergence of Pre-trained Language Models (PLMs) has achieved tremendous success in the field of Natural Language Processing (NLP) by learning universal representations on large corpora in a self-supervised manner. The pre-trained models and the learned representations can be beneficial to a series of downstream NLP tasks. This training paradigm has recently been adapted to the recommendation domain and is considered a promising approach by both academia and industry. In this paper, we systematically investigate how to extract and transfer knowledge from pre-trained models learned by different PLM-related training paradigms to improve recommendation performance from various perspectives, such as generality, sparsity, efficiency and effectiveness. Specifically, we propose a comprehensive taxonomy to divide existing PLM-based recommender systems w.r.t. their training strategies and objectives. Then, we analyze and summarize the connection between PLM-based training paradigms and different input data types for recommender systems. Finally, we elaborate on open issues and future research directions in this vibrant field.
FFHR: Fully and Flexible Hyperbolic Representation for Knowledge Graph Completion
Authors: Wentao Shi, Junkang Wu, Xuezhi Cao, Jiawei Chen, Wenqiang Lei, Wei Wu, Xiangnan He
Date: 2023-02-07 00:00:00
Tasks: Knowledge Graph Completion, Representation Learning

Learning hyperbolic embeddings for knowledge graph (KG) has gained increasing attention due to its superiority in capturing hierarchies. However, some important operations in hyperbolic space still lack good definitions, making existing methods unable to fully leverage the merits of hyperbolic space. Specifically, they suffer from two main limitations: 1) existing Graph Convolutional Network (GCN) methods in hyperbolic space rely on tangent space approximation, which would incur approximation error in representation learning, and 2) due to the lack of inner product operation definition in hyperbolic space, existing methods can only measure the plausibility of facts (links) with hyperbolic distance, which is difficult to capture complex data patterns. In this work, we contribute: 1) a Full Poincar\'{e} Multi-relational GCN that achieves graph information propagation in hyperbolic space without requiring any approximation, and 2) a hyperbolic generalization of Euclidean inner product that is beneficial to capture both hierarchical and complex patterns. On this basis, we further develop a \textbf{F}ully and \textbf{F}lexible \textbf{H}yperbolic \textbf{R}epresentation framework (\textbf{FFHR}) that is able to transfer recent Euclidean-based advances to hyperbolic space. We demonstrate it by instantiating FFHR with four representative KGC methods. Extensive experiments on benchmark datasets validate the superiority of our FFHRs over their Euclidean counterparts as well as state-of-the-art hyperbolic embedding methods.
MDA-Net: Multiscale dual attention-based network for breast lesion segmentation using ultrasound images
Authors: Ahmed Iqbal, Muhammad Sharif
Date: 2021-10-21 00:00:00
Proceeding: Journal of King Saud University – Computer and Information Sciences 2021 10
Tasks: Lesion Segmentation, Segmentation

Accurate breast lesion segmentation is a great help in the initial stage of breast cancer treatment planning. Ultrasound is considered the safe and cheapest method for the breast screening process. However, ultrasound images inherently contain speckle noise, unclear boundaries, and complex shapes, making it more challenging for automatic segmentation methods. This work proposes a multiscale dual attention-based network (MDA-Net) for concurrent segmentation of breast lesions images. The multiscale fusion (MF) block is introduced that addresses the classical fixed receptive field issues, and helps to extract more semantic features and aims to achieve more features diversity. A dual-attention (dA) is also proposed, which is a hybrid of channel-based attention (cA) and lesion attention (lA) blocks that improves the feature representation capability and adaptatively learns a discriminative representation of high-level features. As a result, a combination of two attention blocks helped the proposed network to concentrate on a more relevant field of view of targets. The MDA-Net is extensively tested on both self-collected private datasets and two public UDIAT, BUSIS datasets. Furthermore, our method is also evaluated on MRI datasets to observe the broad applicability of our method in a different imaging modality. The MDA-Net has achieved the DSC of 87.68%, 91.85%, 90.41%, 83.47% on UDIAT, BUSIS, Private, and RIDER breast MRI datasets (p-value < 0.05 with paired t-test). Our MDA-Net implementation code and pretrained models are released at GitHub: https://github.com/ahmedeqbal/MDA-Net
FF-UNet: a U-Shaped Deep Convolutional Neural Network for Multimodal Biomedical Image Segmentation
Authors: Ahmed Iqbal, Muhammad Sharif, Muhammad Attique Khan, Wasif Nisar, Majed Alhaisoni
Date: 2022-06-27 00:00:00
Proceeding: Cognitive Computation 2022 6
Tasks: Image Segmentation, Medical Image Segmentation, Semantic Segmentation

Automatic multimodal image segmentation is considered a challenging research area in the biomedical field. U-shaped models have led to an enormous breakthrough in a large domain of medical image segmentation in recentyears. The receptive field plays an essential role in convolutionalneural networks because too small a receptive field limits context information, and too large loses localization accuracy. Despite outstanding overall performance in biomedical segmenting, classical UNet architecture uses a fixed receptive field in convolutions operations. This study proposes a few modifications in classical UNet architecture by adjusting the receptive field via feature-fused module and attention gate mechanism. Compared with baseline UNet, the numerical parameters of FF-UNet (3.94 million) is 51% of classical UNet architecture (7.75 million). Furthermore, we extended our model performance by introducing post-processing schemes. The tri-threshold fuzzy intensification-based contrast enhancement technique is utilized to improve the contrast of biomedical datasets. In the second tier, the black top-hat filtering-based method is employed to remove hair-like artifacts from the ISIC 2018 skin lesion dataset, which may create a barrier to correctly segmenting the images. The proposed models have been trained using fivefold cross-validation on five publicly available biomedical datasets and achieved the dice coefficients of 0.860, 0.932, 0.932, 0.925, and 0.894 on ETIS-LaribPolypDB, CVC-ColonDB, CVC-ClinicDB, DSB 2018, and ISIC 2018 datasets, respectively. To further verify our claims, comparative analysis based on dice results is conducted, proving the proposed model effectiveness. The FF-UNet implementation models and pre-trained weights are freely publicly available: https://github.com/ahmedeqbal/FF-UNet
Dynamic Programming for Pure-Strategy Subgame Perfection in an Arbitrary Game
Authors: Peter A. Streufert
Date: 2023-02-08 00:00:00
Tasks: Form

This paper uses value functions to characterize the pure-strategy subgame-perfect equilibria of an arbitrary, possibly infinite-horizon game. It specifies the game's extensive form as a pentaform (Streufert 2023p, arXiv:2107.10801v4), which is a set of quintuples formalizing the abstract relationships between nodes, actions, players, and situations (situations generalize information sets). Because a pentaform is a set, this paper can explicitly partition the game form into piece forms, each of which starts at a (Selten) subroot and contains all subsequent nodes except those that follow a subsequent subroot. Then the set of subroots becomes the domain of a value function, and the piece-form partition becomes the framework for a value recursion which generalizes the Bellman equation from dynamic programming. The main results connect the value recursion with the subgame-perfect equilibria of the original game, under the assumptions of upper- and lower-convergence. Finally, a corollary characterizes subgame perfection as the absence of an improving one-piece deviation.
Assigning Optimal Integer Harmonic Periods to Hard Real Time Tasks
Authors: Anand Bhat, Ragunathan Rajkumar
Date: 2023-02-07 00:00:00

Selecting period values for tasks is a very important step in the design process of a real-time system, especially due to the significance of its impact on system schedulability. It is well known that, under RMS, the utilization bound for a harmonic task set is 100%. Also, polynomial-time algorithms have been developed for response-time analysis of harmonic task sets. In practice, the largest acceptable value for the period of a task is determined by the performance and safety requirements of the application. In this paper, we address the problem of assigning harmonic periods to a task set such that every task gets assigned an integer period less than or equal to its application specified upper bound and the task utilization of every task is less than 1. We focus on integer solutions given the discrete nature of time in real-time computer systems. We first express this problem of assigning harmonic periods to a task set as a discrete piecewise optimization problem. We then present the 'Discrete Piecewise Harmonic Search' (DPHS) algorithm that outputs an optimal harmonic task assignment. We then define conditions for a metric to be rational for harmonization. We show that commonly used metrics like, the total percentage error (TPE), total system utilization (TSU), first order error (FOE), and maximum percentage error (MPE), are rational. We next prove that the DPHS algorithm finds the optimal feasible assignment, if one exists, for these rational metrics. We apply the DPHS algorithm to harmonize task sets used in real-world applications to highlight its benefits. We compare the performance of the DPHS algorithm against a brute-force search and find that the DPHS searches up to 94\% fewer task sets than the brute-force search that obtains the optimal solution.
Task Space Control of Robot Manipulators based on Visual SLAM
Authors: Seyed Hamed Hashemi, Jouni Mattila
Date: 2023-02-08 00:00:00
Tasks: LEMMA, Simultaneous Localization and Mapping

This paper aims to address the open problem of designing a globally stable vision-based controller for robot manipulators. Accordingly, based on a hybrid mechanism, this paper proposes a novel task-space control law attained by taking the gradient of a potential function in SE(3). The key idea is to employ the Visual Simultaneous Localization and Mapping (VSLAM) algorithm to estimate a robot pose. The estimated robot pose is then used in the proposed hybrid controller as feedback information. Invoking Barbalats lemma and Lyapunov's stability theorem, it is guaranteed that the resulting closed-loop system is globally asymptotically stable, which is the main accomplishment of the proposed structure. Simulation studies are conducted on a six degrees of freedom (6-DOF) robot manipulator to demonstrate the effectiveness and validate the performance of the proposed VSLAM-based control scheme.
Masking Kernel for Learning Energy-Efficient Representations for Speaker Recognition and Mobile Health
Authors: Apiwat Ditthapron, Emmanuel O. Agu, Adam C. Lammert
Date: 2023-02-08 00:00:00
Tasks: Speaker Recognition

Modern smartphones possess hardware for audio acquisition and to perform speech processing tasks such as speaker recognition and health assessment. However, energy consumption remains a concern, especially for resource-intensive DNNs. Prior work has improved the DNN energy efficiency by utilizing a compact model or reducing the dimensions of speech features. Both approaches reduced energy consumption during DNN inference but not during speech acquisition. This paper proposes using a masking kernel integrated into gradient descent during DNN training to learn the most energy-efficient speech length and sampling rate for windowing, a common step for sample construction. To determine the most energy-optimal parameters, a masking function with non-zero derivatives was combined with a low-pass filter. The proposed approach minimizes the energy consumption of both data collection and inference by 57%, and is competitive with speaker recognition and traumatic brain injury detection baselines.
Definition of Static and Dynamic Load Models for Grid Studies of Electric Vehicles Connected to Fast Charging Stations
Authors: Davide del Giudice, Angelo Maurizio Brambilla, Federico Bizzarri, Daniele Linaro, Samuele Grillo
Date: 2023-02-08 00:00:00
Tasks: CPU

The growing deployment of electric mobility calls for power system analyses to investigate to what extent the simultaneous charging of electric vehicles leads to degraded network operation and to validate the efficiency of countermeasures. To reduce complexity and CPU time, a common approach while performing these analyses consists in replacing electric vehicles and their charging stations with constant PQ loads. However, this approach is inaccurate, as the power absorbed by these elements actually depends not only on voltage but also on the state of charge, charging method, cathode chemistry of the battery pack, and converter controls in the electric vehicle and charging station. By considering all these aspects, this article develops a novel static load model and a vector fitting-based dynamic load model for electric vehicles connected to fast charging stations. These computationally efficient representations can replace the standard constant PQ load model of electric vehicles to assess more accurately their impact in static and dynamic grid studies. Simulation results of the IEEE14 system modified by adding fleets of electric vehicles prove the accuracy of the proposed models and highlight the shortcomings of the standard electric vehicle representation as a constant PQ load in some cases.