pwc-archive/papers-with-abstracts huggingface dataset:
User behavior understanding in real world settings
Authors: Weiqi Shao, Xu Chen, Jiashu Zhao, Long Xia, Dawei Yin
Date: 2021-12-06 00:00:00

How to extract meaningful information in user historical behavior plays a crucial role in recommendation. User behavior sequence often contains multiple conceptually distinct items that belong to different item groups and the number of the item groups is changing over time. It is necessary to learn a dynamic group of representations according the item groups in a user historical behavior. However, current works only learns a predefined and fixed number representations which includes single representation methods and multi representations methods from the user context that could lead to suboptimal recommendation quality. In this paper we propose a model that can automatically and adaptively generates a dynamic group of representations from the user behavior accordingly. To be specific, AutoRep is composed of an informative representation construct (IRC) module and a dynamic representations construct (DRC) module. The IRC module learns the overall sequential characteristics of user behavior with a bi-directional architecture transformer. The DRC module dynamically allocate the item in the user behavior into different item groups and form a dynamic group of representations in a differentiable method. Such design improves the model recommendation performance. We evaluate the proposed model on five benchmark datasets. The results show that AutoRep outperforms representative baselines. Further ablation study has been conducted to deepen our understandings of AutoRep, including the proposed module IRC and DRC.
Gumble Softmax For User Behavior Modeling
Authors: Weiqi Shao, Xu Chen, Jiashu Zhao, Long Xia, Dawei Yin
Date: 2021-12-06 00:00:00
Tasks: Recommendation Systems, Sequential Recommendation

Recently, sequential recommendation systems are important in solving the information overload in many online services. Current methods in sequential recommendation focus on learning a fixed number of representations for each user at any time, with a single representation or multi representations for the user. However, when a user is exploring items on an e-commerce recommendation system, the number of this user's hobbies may change overtime (e.g. increase/reduce one more interest), affected by the user's evolving self needs. Moreover, different users may have various number of interests. In this paper, we argue that it is meaningful to explore a personalized dynamic number of user interests, and learn a dynamic group of user interest representations accordingly. We propose a sequential model with dynamic number of representations for recommendation systems (RDRSR). Specifically, RDRSR is composed of a dynamic interest discriminator (DID) module and a dynamic interest allocator (DIA) module. The DID module explores the number of a user's interests by learning the overall sequential characteristics with bi-directional self-attention and Gumbel-Softmax. The DIA module make the historical clicked items into a group of item groups and constructs user's dynamic interest representation. Additionally, experiments on the real-world datasets demonstrates our model's effectiveness.
Variational Autoencoder with CCA for Audio-Visual Cross-Modal Retrieval
Authors: Jiwei Zhang, Yi Yu, Suhua Tang, Jianming Wu, Wei Li
Date: 2021-12-05 00:00:00
Tasks: Cross-Modal Retrieval, Information Retrieval, Retrieval

Cross-modal retrieval is to utilize one modality as a query to retrieve data from another modality, which has become a popular topic in information retrieval, machine learning, and database. How to effectively measure the similarity between different modality data is the major challenge of cross-modal retrieval. Although several reasearch works have calculated the correlation between different modality data via learning a common subspace representation, the encoder's ability to extract features from multi-modal information is not satisfactory. In this paper, we present a novel variational autoencoder (VAE) architecture for audio-visual cross-modal retrieval, by learning paired audio-visual correlation embedding and category correlation embedding as constraints to reinforce the mutuality of audio-visual information. On the one hand, audio encoder and visual encoder separately encode audio data and visual data into two different latent spaces. Further, two mutual latent spaces are respectively constructed by canonical correlation analysis (CCA). On the other hand, probabilistic modeling methods is used to deal with possible noise and missing information in the data. Additionally, in this way, the cross-modal discrepancy from intra-modal and inter-modal information are simultaneously eliminated in the joint embedding subspace. We conduct extensive experiments over two benchmark datasets. The experimental outcomes exhibit that the proposed architecture is effective in learning audio-visual correlation and is appreciably better than the existing cross-modal retrieval methods.
Long-Tail Session-based Recommendation from Calibration
Authors: Jiayi Chen, Wen Wu, Wei Zheng, Liang He
Date: 2021-12-05 00:00:00
Tasks: Recommendation Systems, Session-Based Recommendations

Accurate predictions in session-based recommendations have progressed, but a few studies have focused on skewed recommendation lists caused by popularity bias. Existing models for mitigating popularity bias have attempted to reduce the overconcentration of popular items by amplifying scores of less popular items. However, they normally ignore the users' different preferences toward long-tail items. Thus, we incorporate calibration, where calibrated recommendations reflect the users' interests in recommendation lists with appropriate proportions, to mitigate the popularity bias from the user's perspective. Specifically, we propose a calibration module to predict the ratio of tail items in the recommendation list from the session representation, and align it to the ongoing session. Additionally, we utilize a two-stage curriculum training strategy to improve prediction in the calibration module. Experiments on benchmark datasets show that our model can both achieve the competitive accuracy of recommendation and provide more tail items.
Extracting and Measuring Uncertain Biomedical Knowledge from Scientific Statements
Authors: Xin Guo, Yuming Chen, Jian Du, Erdan Dong
Date: 2021-12-05 00:00:00
Tasks: Diagnostic

Purpose: This study aims to develop a novel approach to extracting and measuring uncertain biomedical knowledge from scientific statements. Design/methodology/approach: Taking cardiovascular research publications in China as a sample, we extracted the SPO triples as knowledge unit and the hedging/conflicting uncertainties as the knowledge context. We introduced Information Entropy and Uncertainty Rate as potential metrics to quantity the uncertainty of biomedical knowledge claims represented at different levels, such as the SPO triples (micro level), as well as the semantic type pairs (micro-level). Findings: The results indicated that while the number of scientific publications and total SPO triples showed a liner growth, the novel SPO triples occurring per year remained stable. After examining the frequency of uncertain cue words in different part of scientific statements, we found hedging words tend to appear in conclusive and purposeful sentences, whereas conflicting terms often appear in background and act as the premise (e.g., unsettled scientific issues) of the work to be investigated. Practical implications: Our approach identified major uncertain knowledge areas, such as diagnostic biomarkers, genetic characteristics, and pharmacologic therapies surrounding cardiovascular diseases in China. These areas are suggested to be prioritized in which new hypotheses need to be verified, and disputes, conflicts, as well as contradictions to be settled further.
A comment-driven evidence appraisal approach for decision-making when only uncertain evidence available
Authors: Shuang Wang, Jian Du
Date: 2021-12-05 00:00:00
Tasks: Decision Making

Purpose: To explore whether comments could be used as an assistant tool for heuristic decision-making, especially in cases where missing, incomplete, uncertain, or even incorrect evidence is acquired. Methods: Six COVID-19 drug candidates were selected from WHO clinical guidelines. Evidence-comment networks (ECNs) were completed of these six drug candidates based on evidence-comment pairs from all PubMed indexed COVID-19 publications with formal published comments. WHO guidelines were utilized to validate the feasibility of comment-derived evidence assertions as a fast decision supporting tool. Results: Out of 6 drug candidates, comment-derived evidence assertions of leading subgraphs of 5 drugs were consistent with WHO guidelines, and the overall comment sentiment of 6 drugs was aligned with WHO clinical guidelines. Additionally, comment topics were in accordance with the concerns of guidelines and evidence appraisal criteria. Furthermore, half of the critical comments emerged 4.5 months earlier than the date guidelines were published. Conclusions: Comment-derived evidence assertions have the potential as an evidence appraisal tool for heuristic decisions based on the accuracy, sensitivity, and efficiency of evidence-comment networks. In essence, comments reflect that academic communities do have a self-screening evaluation and self-purification (argumentation) mechanism, thus providing a tool for decision makers to filter evidence.
Exploring and Mitigating Gender Bias in Recommender Systems with Explicit Feedback
Authors: Shrikant Saxena, Shweta Jain
Date: 2021-12-05 00:00:00
Tasks: Recommendation Systems

Recommender systems are indispensable because they influence our day-to-day behavior and decisions by giving us personalized suggestions. Services like Kindle, Youtube, and Netflix depend heavily on the performance of their recommender systems to ensure that their users have a good experience and to increase revenues. Despite their popularity, it has been shown that recommender systems reproduce and amplify the bias present in the real world. The resulting feedback creates a self-perpetuating loop that deteriorates the user experience and results in homogenizing recommendations over time. Further, biased recommendations can also reinforce stereotypes based on gender or ethnicity, thus reinforcing the filter bubbles that we live in. In this paper, we address the problem of gender bias in recommender systems with explicit feedback. We propose a model to quantify the gender bias present in book rating datasets and in the recommendations produced by the recommender systems. Our main contribution is to provide a principled approach to mitigate the bias being produced in the recommendations. We theoretically show that the proposed approach provides unbiased recommendations despite biased data. Through empirical evaluation on publicly available book rating datasets, we further show that the proposed model can significantly reduce bias without significant impact on accuracy. Our method is model agnostic and can be applied to any recommender system. To demonstrate the performance of our model, we present the results on four recommender algorithms, two from the K-nearest neighbors family, UserKNN and ItemKNN, and the other two from the matrix factorization family, Alternating least square and Singular value decomposition.
Grappling with the Scale of Born-Digital Government Publications: Toward Pipelines for Processing and Searching Millions of PDFs
Authors: Benjamin Charles Germain Lee, Trevor Owens
Date: 2021-12-05 00:00:00

Official government publications are key sources for understanding the history of societies. Web publishing has fundamentally changed the scale and processes by which governments produce and disseminate information. Significantly, a range of web archiving programs have captured massive troves of government publications. For example, hundreds of millions of unique U.S. Government documents posted to the web in PDF form have been archived by libraries to date. Yet, these PDFs remain largely unutilized and understudied in part due to the challenges surrounding the development of scalable pipelines for searching and analyzing them. This paper utilizes a Library of Congress dataset of 1,000 government PDFs in order to offer initial approaches for searching and analyzing these PDFs at scale. In addition to demonstrating the utility of PDF metadata, this paper offers computationally-efficient machine learning approaches to search and discovery that utilize the PDFs' textual and visual features as well. We conclude by detailing how these methods can be operationalized at scale in order to support systems for navigating millions of PDFs.
Considering user dynamic preferences for mitigating negative effects of long tail in recommender systems
Authors: Reza Shafiloo, Marjan Kaedi, Ali Pourmiri
Date: 2021-12-04 00:00:00
Tasks: Recommendation Systems

Nowadays, with the increase in the amount of information generated in the webspace, many web service providers try to use recommender systems to personalize their services and make accessing the content convenient. Recommender systems that only try to increase the accuracy (i.e., the similarity of items to users' interest) will face the long tail problem. It means that popular items called short heads appear in the recommendation lists more than others since they have many ratings. However, unpopular items called long-tail items are used less than popular ones as they reduce accuracy. Other studies that solve the long-tail problem consider users' interests constant while their preferences change over time. We suggest that users' dynamic preferences should be taken into account to prevent the loss of accuracy when we use long-tail items in recommendation lists. This study shows that the two reasons lie in the following: 1) Users rate for different proportions of popular and unpopular items over time. 2) Users of all ages have various interests in popular and unpopular items. As a result, recommendation lists can be created over time with a different portion of long-tail and short-head items. Besides, we predict the age of users based on item ratings to use more long-tail items. The results show that by considering these two reasons, the accuracy of recommendation lists reaches 91%. At the same time, the long tail problem is better improved than other related research and provides better diversity in recommendation lists in the long run.
Recommender systems: when memory matters
Authors: Aleksandra Burashnikova, Marianne Clausel, Massih-Reza Amini, Yury Maximov, Nicolas Dante
Date: 2021-12-04 00:00:00
Tasks: Recommendation Systems

In this paper, we study the effect of long memory in the learnability of a sequential recommender system including users' implicit feedback. We propose an online algorithm, where model parameters are updated user per user over blocks of items constituted by a sequence of unclicked items followed by a clicked one. We illustrate through thorough empirical evaluations that filtering users with respect to the degree of long memory contained in their interactions with the system allows to substantially gain in performance with respect to MAP and NDCG, especially in the context of training large-scale Recommender Systems.
Active Learning Meets Optimized Item Selection
Authors: Bernard Kleynhans, Xin Wang, Serdar Kadıoğlu
Date: 2021-11-22 00:00:00
Tasks: Active Learning, Clustering, Combinatorial Optimization, Recommendation Systems

Designing recommendation systems with limited or no available training data remains a challenge. To that end, a new combinatorial optimization problem is formulated to generate optimized item selection for experimentation with the goal to shorten the time for collecting randomized training data. We first present an overview of the optimized item selection problem and a multi-level optimization framework to solve it. The approach integrates techniques from discrete optimization, unsupervised clustering, and latent text embeddings. We then discuss how to incorporate optimized item selection with active learning as part of randomized exploration in an ongoing fashion.
FastWARC: Optimizing Large-Scale Web Archive Analytics
Authors: Janek Bevendorff, Martin Potthast, Benno Stein
Date: 2021-11-22 00:00:00

Web search and other large-scale web data analytics rely on processing archives of web pages stored in a standardized and efficient format. Since its introduction in 2008, the IIPC's Web ARCive (WARC) format has become the standard format for this purpose. As a list of individually compressed records of HTTP requests and responses, it allows for constant-time random access to all kinds of web data via off-the-shelf open source parsers in many programming languages, such as WARCIO, the de-facto standard for Python. When processing web archives at the terabyte or petabyte scale, however, even small inefficiencies in these tools add up quickly, resulting in hours, days, or even weeks of wasted compute time. Reviewing the basic components of WARCIO and analyzing its bottlenecks, we proceed to build FastWARC, a new high-performance WARC processing library for Python, written in C++/Cython, which yields performance improvements by factors of 1.6-8x.
Extracting Domain-specific Concepts from Large-scale Linked Open Data
Authors: Satoshi Kume, Kouji Kozaki
Date: 2021-11-22 00:00:00

We propose a methodology for extracting concepts for a target domain from large-scale linked open data (LOD) to support the construction of domain ontologies providing field-specific knowledge and definitions. The proposed method defines search entities by linking the LOD vocabulary with technical terms related to the target domain. The search entities are then used as a starting point for obtaining upper-level concepts in the LOD, and the occurrences of common upper-level entities and the chain-of-path relationships are examined to determine the range of conceptual connections in the target domain. A technical dictionary index and natural language processing are used to evaluate whether the extracted concepts cover the domain. As an example of extracting a class hierarchy from LOD, we used Wikidata to construct a domain ontology for polymer materials and physical properties. The proposed method can be applied to general datasets with class hierarchies, and it allows ontology developers to create an initial model of the domain ontology for their own purposes.
Speech emotion recognition with deep convolutional neural networks
Authors: Dias Issa, M. Fatih Demirci, Adnan Yazici
Date: 2020-02-15 00:00:00
Proceeding: Biomedical Signal Processing and Control 2020 2
Tasks: Audio Classification, Emotion Recognition, Speech Emotion Recognition

The speech emotion recognition (or, classification) is one of the most challenging topics in data science. In this work, we introduce a new architecture, which extracts mel-frequency cepstral coefficients, chroma-gram, mel-scale spectrogram, Tonnetz representation, and spectral contrast features from sound files and uses them as inputs for the one-dimensional Convolutional Neural Network for the identification of emotions using samples from the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS),Berlin (EMO-DB), and Interactive Emotional Dyadic Motion Capture (IEMOCAP) datasets. We utilize an incremental method for modifying our initial model in order to improve classification accuracy. All of the proposed models work directly with raw sound data without the need for conversion to visual representations, unlike some previous approaches. Based on experimental results, our best-performing model outperforms existing frameworks for RAVDESS and IEMOCAP, thus setting the new state-of-the-art. For the EMO-DB dataset, it outperforms all previous works except one but compares favorably with that one in terms of generality, simplicity, and applicability. Specifically, the proposed framework obtains 71.61%for RAVDESS with 8 classes, 86.1% for EMO-DB with 535 samples in 7 classes, 95.71% for EMO-DB with 520samples in 7 classes, and 64.3% for IEMOCAP with 4 classes in speaker-independent audio classification tasks.
Logic Shrinkage: Learned FPGA Netlist Sparsity for Efficient Neural Network Inference
Authors: Erwei Wang, James J. Davis, Georgios-Ilias Stavrou, Peter Y. K. Cheung, George A. Constantinides, Mohamed S. Abdelfattah
Date: 2021-12-04 00:00:00
Tasks: Efficient Neural Network, GPU

FPGA-specific DNN architectures using the native LUTs as independently trainable inference operators have been shown to achieve favorable area-accuracy and energy-accuracy tradeoffs. The first work in this area, LUTNet, exhibited state-of-the-art performance for standard DNN benchmarks. In this paper, we propose the learned optimization of such LUT-based topologies, resulting in higher-efficiency designs than via the direct use of off-the-shelf, hand-designed networks. Existing implementations of this class of architecture require the manual specification of the number of inputs per LUT, K. Choosing appropriate K a priori is challenging, and doing so at even high granularity, e.g. per layer, is a time-consuming and error-prone process that leaves FPGAs' spatial flexibility underexploited. Furthermore, prior works see LUT inputs connected randomly, which does not guarantee a good choice of network topology. To address these issues, we propose logic shrinkage, a fine-grained netlist pruning methodology enabling K to be automatically learned for every LUT in a neural network targeted for FPGA inference. By removing LUT inputs determined to be of low importance, our method increases the efficiency of the resultant accelerators. Our GPU-friendly solution to LUT input removal is capable of processing large topologies during their training with negligible slowdown. With logic shrinkage, we better the area and energy efficiency of the best-performing LUTNet implementation of the CNV network classifying CIFAR-10 by 1.54x and 1.31x, respectively, while matching its accuracy. This implementation also reaches 2.71x the area efficiency of an equally accurate, heavily pruned BNN. On ImageNet with the Bi-Real Net architecture, employment of logic shrinkage results in a post-synthesis area reduction of 2.67x vs LUTNet, allowing for implementation that was previously impossible on today's largest FPGAs.
A Methodology for Thermal Simulation of Interconnects Enabled by Model Reduction with Material Property Variation
Authors: Wangkun Jia, Ming-C. Cheng
Date: 2021-12-01 00:00:00

A thermal simulation methodology is developed for interconnects enabled by a data-driven learning algorithm accounting for variations of material properties, heat sources and boundary conditions (BCs). The methodology is based on the concepts of model order reduction and domain decomposition to construct a multi-block approach. A generic block model is built to represent a group of interconnect blocks that are used to wire standard cells in the integrated circuits (ICs). The blocks in this group possess identical geometry with various metal/via routings. The data-driven model reduction method is thus applied to learn material property variations induced by different metal/via routings in the blocks, in addition to the variations of heat sources and BCs. The approach is investigated in two very different settings. It is first applied to thermal simulation of a single interconnect block with similar BCs to those in the training of the generic block. It is then implemented in multi-block thermal simulation of a FinFET IC, where the interconnect structure is partitioned into several blocks each modeled by the generic block model. Accuracy of the generic block model is examined in terms of the metal/via routings, BCs and thermal discontinuities at the block interfaces.