pwc-archive/papers-with-abstracts huggingface dataset:
Modular Retrieval for Generalization and Interpretation
Authors: Juhao Liang, Chen Zhang, Zhengyang Tang, Jie Fu, Dawei Song, Benyou Wang
Date: 2023-03-23 00:00:00
Tasks: Language Modeling, Language Modelling, Retrieval

New retrieval tasks have always been emerging, thus urging the development of new retrieval models. However, instantiating a retrieval model for each new retrieval task is resource-intensive and time-consuming, especially for a retrieval model that employs a large-scale pre-trained language model. To address this issue, we shift to a novel retrieval paradigm called modular retrieval, which aims to solve new retrieval tasks by instead composing multiple existing retrieval modules. Built upon the paradigm, we propose a retrieval model with modular prompt tuning named REMOP. It constructs retrieval modules subject to task attributes with deep prompt tuning, and yields retrieval models subject to tasks with module composition. We validate that, REMOP inherently with modularity not only has appealing generalizability and interpretability in preliminary explorations, but also achieves comparable performance to state-of-the-art retrieval models on a zero-shot retrieval benchmark.\footnote{Our code is available at \url{https://github.com/FreedomIntelligence/REMOP}}
A Unified Framework for Learned Sparse Retrieval
Authors: Thong Nguyen, Sean MacAvaney, Andrew Yates
Date: 2023-03-23 00:00:00
Tasks: Retrieval

Learned sparse retrieval (LSR) is a family of first-stage retrieval methods that are trained to generate sparse lexical representations of queries and documents for use with an inverted index. Many LSR methods have been recently introduced, with Splade models achieving state-of-the-art performance on MSMarco. Despite similarities in their model architectures, many LSR methods show substantial differences in effectiveness and efficiency. Differences in the experimental setups and configurations used make it difficult to compare the methods and derive insights. In this work, we analyze existing LSR methods and identify key components to establish an LSR framework that unifies all LSR methods under the same perspective. We then reproduce all prominent methods using a common codebase and re-train them in the same environment, which allows us to quantify how components of the framework affect effectiveness and efficiency. We find that (1) including document term weighting is most important for a method's effectiveness, (2) including query weighting has a small positive impact, and (3) document expansion and query expansion have a cancellation effect. As a result, we show how removing query expansion from a state-of-the-art model can reduce latency significantly while maintaining effectiveness on MSMarco and TripClick benchmarks. Our code is publicly available at https://github.com/thongnt99/learned-sparse-retrieval
Limits of Predictability in Top-N Recommendation
Authors: En Xu, Zhiwen Yu, Ying Zhang, Bin Guo, Lina Yao
Date: 2023-03-23 00:00:00

Top-N recommendation aims to recommend each consumer a small set of N items from a large collection of items, and its accuracy is one of the most common indexes to evaluate the performance of a recommendation system. While a large number of algorithms are proposed to push the Top-N accuracy by learning the user preference from their history purchase data, a predictability question is naturally raised - whether there is an upper limit of such Top-N accuracy. This work investigates such predictability by studying the degree of regularity from a specific set of user behavior data. Quantifying the predictability of Top-N recommendations requires simultaneously quantifying the limits on the accuracy of the N behaviors with the highest probability. This greatly increases the difficulty of the problem. To achieve this, we firstly excavate the associations among N behaviors with the highest probability and describe the user behavior distribution based on the information theory. Then, we adopt the Fano inequality to scale and obtain the Top-N predictability. Extensive experiments are conducted on the real-world data where significant improvements are observed compared to the state-of-the-art methods. We have not only completed the predictability calculation for N targets but also obtained predictability that is much closer to the true value than existing methods. We expect our results to assist these research areas where the quantitative requirement of Top-N predictability is required.
Reinforcement Learning Enhanced PicHunter for Interactive Search
Authors: Zhixin Ma, Jiaxin Wu, Weixiong Loo, and Chong-Wah Ngo
Date: 2023-01-09 00:00:00
Proceeding: 29th International Conference on Multimedia Modeling 2023 1
Tasks: Bayesian Inference, reinforcement-learning, Reinforcement Learning

With the tremendous increase in video data size, search performance could be impacted significantly. Specifically, in an interactive system, a real-time system allows a user to browse, search and refine a query. Without a speedy system quickly, the main ingredient to engage a user to stay focused, an interactive system becomes less effective even with a sophisticated deep learning system. This paper addresses this challenge by leveraging approximate search, Bayesian inference, and reinforcement learning. For approximate search, we apply a hierarchical navigable small world, which is an efficient approximate nearest neighbor search algorithm. To quickly prune the search scope, we integrate PicHunter, one of the most popular engines in Video Browser Showdown, with reinforcement learning. The integration enhances PicHunter with the ability of systematic planning. Specifically, PicHunter performs a Bayesian update with a greedy strategy to select a small number of candidates for display. With reinforcement learning, the greedy strategy is replaced with a policy network that learns to select candidates that will result in the minimum number of user iterations, which is analytically defined by a reward function. With these improvements, the interactive system only searches a subset of video datasets relevant to a query while being able to quickly perform Bayesian updates with systematic planning to recommend the most probable candidates that can potentially lead to minimum iteration rounds.
A novel facial emotion recognition model using segmentation VGG-19 architecture
Authors: S. Vignesh, M. Savithadevi, M. Sridevi, Rajeswari Sridhar
Date: 2023-03-24 00:00:00
Proceeding: International Journal of Information Technology 2023 3
Tasks: Emotion Recognition, Facial Emotion Recognition, Facial Expression Recognition (FER), feature selection

Facial Emotion Recognition (FER) has gained popularity in recent years due to its many applications, including biometrics, detection of mental illness, understanding of human behavior, and psychological profiling. However, developing an accurate and robust FER pipeline is still challenging because multiple factors make it difficult to generalize across different emotions. The factors that challenge a promising FER pipeline include pose variation, heterogeneity of the facial structure, illumination, occlusion, low resolution, and aging factors. Many approaches were developed to overcome the above problems, such as the Histogram of Oriented Gradients (HOG) and Local Binary Pattern (LBP) histogram. However, these methods require manual feature selection. Convolutional Neural Networks (CNN) overcame this manual feature selection problem. CNN has shown great potential in FER tasks due to its unique feature extraction strategy compared to regular FER models. In this paper, we propose a novel CNN architecture by interfacing U-Net segmentation layers in-between Visual Geometry Group (VGG) layers to allow the network to emphasize more critical features from the feature map, which also controls the flow of redundant information through the VGG layers. Our model achieves state-of-the-art (SOTA) single network accuracy compared with other well-known FER models on the FER-2013 dataset.
TCMPR: TCM Prescription recommendation based on subnetwork term mapping and deep learning
Authors: Xin Dong, Yi Zheng, Zixin Shu, Kai Chang, Dengying Yan, Jianan Xia, Qiang Zhu, Kunyu Zhong, Xinyan Wang, Kuo Yang, Xuezhong Zhou
Date: 2021-12-09 00:00:00
Proceeding: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) 2021 12

Traditional Chinese medicine (TCM) has played an indispensable role in clinical diagnose and treatment. Based on patient’s symptom phenotypes, computation-based prescription recommendation methods can recommend personalized TCM prescription using machine learning and artificial intelligence technologies. However, owing to the complexity and individuation of patient’s clinical phenotypes, current prescription recommendation methods cannot obtain good performance. Meanwhile, it’s very difficult to conduct effective representation for unrecorded symptom terms in existing knowledge base. In this study, we proposed a subnetwork-based symptom term mapping method (SSTM), and constructed a SSTM-based TCM prescription recommendation method (termed TCMPR). Our SSTM can extract the subnetwork structure between symptoms from knowledge network to effectively represent the embedding features of clinical symptom terms (especially, the unrecorded terms). The experimental results showed that our method performs better than state-of-the-art methods. In addition, the comprehensive experiments of TCMPR with different hyper parameters (i.e., feature embedding, feature dimension and feature fusion) that demonstrates that our method has high performance on TCM prescription recommendation and potentially promote clinical diagnosis and treatment of TCM precision medicine.
Accurate deep learning off-target prediction with novel sgRNA-DNA sequence encoding in CRISPR-Cas9 gene editing
Authors: Jeremy Charlier, Robert Nadon, Vladimir Makarenkov
Date: 2021-02-18 00:00:00
Proceeding: 10.1093/bioinformatics/btab112 2021 2
Tasks: Prediction

Motivation: Off-target predictions are crucial in gene editing research. Recently, significant progress has been made in the field of prediction of off-target mutations, particularly with CRISPR-Cas9 data, thanks to the use of deep learning. CRISPR-Cas9 is a gene editing technique which allows manipulation of DNA fragments. The sgRNA-DNA (single guide RNA-DNA) sequence encoding for deep neural networks, however, has a strong impact on the prediction accuracy. We propose a novel encoding of sgRNA-DNA sequences that aggregates sequence data with no loss of information.

Results: In our experiments, we compare the proposed sgRNA-DNA sequence encoding applied in a deep learning prediction framework with state-of-the-art encoding and prediction methods. We demonstrate the superior accuracy of our approach in a simulation study involving Feedforward Neural Networks (FNNs), Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) as well as the traditional Random Forest (RF), Naive Bayes (NB) and Logistic Regression (LR) classifiers.We highlight the quality of our results by building several FNNs, CNNs and RNNs with various layer depths and performing predictions on two popular CRISPOR and GUIDE-seq gene editing data sets. In all our experiments, the new encoding led to more accurate off-target prediction results, providing an improvement of the area under the Receiver Operating Characteristic (ROC) curve up to 35%.
Field-aware Variational Autoencoders for Billion-scale User Representation Learning
Authors: Ge Fan, Chaoyun Zhang, Junyang Chen, Baopu Li, Zenglin Xu, Yingjie Li, Luyu Peng, Zhiguo Gong
Date: 2022-08-01 00:00:00
Proceeding: IEEE 38th International Conference on Data Engineering (ICDE) 2022 8
Tasks: Recommendation Systems, Representation Learning, TAG

User representation learning plays an essential role in Internet applications, such as recommender systems. Though developing a universal embedding for users is demanding, only few previous works are conducted in an unsupervised learning manner. The unsupervised method is however important as most of the user data is collected without specific labels. In this paper, we harness the unsupervised advantages of Variational Autoencoders (VAEs), to learn user representation from large-scale, high-dimensional, and multi-field data. We extend the traditional VAE by developing Field-aware VAE (FVAE) to model each feature field with an independent multinomial distribution. To reduce the complexity in training, we employ dynamic hash tables, a batched softmax function, and a feature sampling strategy to improve the efficiency of our method. We conduct experiments on multiple datasets, showing that the proposed FVAE significantly outperforms baselines on several tasks of data reconstruction and tag prediction. Moreover, we deploy the proposed method in real-world applications and conduct online A/B tests in a look-alike system. Results demonstrate that our method can effectively improve the quality of recommendation. To the best of our knowledge, it is the first time that the VAE-based user representation learning model is applied to real-world recommender systems.
Collaborative filtering via heterogeneous neural networks
Authors: Ge Fan, Biao Geng, Jianrong Tao, Kai Wang, Changjie Fan, Wei Zeng
Date: 2022-02-01 00:00:00
Proceeding: Neurocomputing 2022 2
Tasks: Collaborative Filtering, Informativeness, Link Prediction, Network Embedding

After being proved extremely useful in many applications, the network embedding has played a critical role in the network analysis. Most of recent works usually model the network by minimizing the joint probability that the target node co-occurs with its neighboring nodes. These methods may fail to capture the personalized informativeness of each vertex. In this work, we propose a method named Personalized Proximity Preserved Network Embedding (PPPNE) to adaptively capture the personalization of vertices based on the personalized ranking loss. Our theoretical analysis shows that PPPNE generalizes prior work based on the matrix factorization or the neural network with a single layer, and we argue that preserving personalized proximity is the key to learning more informative representations. Moreover, to better capture the network structure in multiple scales, we exploit the distance ordering of each vertex. Our method can be efficiently optimized with a vertex-anchored sampling strategy. The results of extensive experiments on five real-world networks demonstrate that our approach outperforms state-of-the-art network embedding methods with a considerable improvement on several common tasks including link prediction and vertex classification. Additionally, PPPNE is efficient and can be easily accelerated by parallel computing, which enables PPPNE to work on large-scale networks.
PPPNE: Personalized proximity preserved network embedding
Authors: Ge Fan, Biao Geng, Jianrong Tao, Kai Wang, Changjie Fan, Wei Zeng
Date: 2022-02-01 00:00:00
Proceeding: Neurocomputing 2022 2
Tasks: Graph Embedding, Informativeness, Link Prediction, Network Embedding

After being proved extremely useful in many applications, the network embedding has played a critical role in the network analysis. Most of recent works usually model the network by minimizing the joint probability that the target node co-occurs with its neighboring nodes. These methods may fail to capture the personalized informativeness of each vertex. In this work, we propose a method named Personalized Proximity Preserved Network Embedding (PPPNE) to adaptively capture the personalization of vertices based on the personalized ranking loss. Our theoretical analysis shows that PPPNE generalizes prior work based on the matrix factorization or the neural network with a single layer, and we argue that preserving personalized proximity is the key to learning more informative representations. Moreover, to better capture the network structure in multiple scales, we exploit the distance ordering of each vertex. Our method can be efficiently optimized with a vertex-anchored sampling strategy. The results of extensive experiments on five real-world networks demonstrate that our approach outperforms state-of-the-art network embedding methods with a considerable improvement on several common tasks including link prediction and vertex classification. Additionally, PPPNE is efficient and can be easily accelerated by parallel computing, which enables PPPNE to work on large-scale networks.
BART-IT: An Efficient Sequence-to-Sequence Model for Italian Text Summarization
Authors: La Quatra Moreno, Cagliero Luca
Date: 2022-12-27 00:00:00
Proceeding: Future Internet 2022 12
Tasks: Abstractive Text Summarization, Text Summarization

The emergence of attention-based architectures has led to significant improvements in the performance of neural sequence-to-sequence models for text summarization. Although these models have proved to be effective in summarizing English-written documents, their portability to other languages is limited thus leaving plenty of room for improvement. In this paper, we present BART-IT, a sequence-to-sequence model, based on the BART architecture that is specifically tailored to the Italian language. The model is pre-trained on a large corpus of Italian-written pieces of text to learn language-specific features and then fine-tuned on several benchmark datasets established for abstractive summarization. The experimental results show that BART-IT outperforms other state-of-the-art models in terms of ROUGE scores in spite of a significantly smaller number of parameters. The use of BART-IT can foster the development of interesting NLP applications for the Italian language. Beyond releasing the model to the research community to foster further research and applications, we also discuss the ethical implications behind the use of abstractive summarization models.
Optimal Security Parameter for Encrypted Control Systems Against Eavesdropper and Malicious Server
Authors: Kaoru Teranishi, Kiminao Kogiso
Date: 2023-03-23 00:00:00

A sample identifying complexity and a sample deciphering time have been introduced in a previous study to capture an estimation error and a computation time of system identification by adversaries. The quantities play a crucial role in defining the security of encrypted control systems and designing a security parameter. This study proposes an optimal security parameter for an encrypted control system under a network eavesdropper and a malicious controller server who attempt to identify system parameters using a least squares method. The security parameter design is achieved based on a modification of conventional homomorphic encryption for improving a sample deciphering time and a novel sample identifying complexity, characterized by controllability Gramians and the variance ratio of identification input to system noise. The effectiveness of the proposed design method for a security parameter is demonstrated through numerical simulations.
DRSM: De-Randomized Smoothing on Malware Classifier Providing Certified Robustness
Authors: Shoumik Saha, Wenxiao Wang, Yigitcan Kaya, Soheil Feizi, Tudor Dumitras
Date: 2023-03-20 00:00:00
Tasks: Adversarial Robustness, Malware Detection

Machine Learning (ML) models have been utilized for malware detection for over two decades. Consequently, this ignited an ongoing arms race between malware authors and antivirus systems, compelling researchers to propose defenses for malware-detection models against evasion attacks. However, most if not all existing defenses against evasion attacks suffer from sizable performance degradation and/or can defend against only specific attacks, which makes them less practical in real-world settings. In this work, we develop a certified defense, DRSM (De-Randomized Smoothed MalConv), by redesigning the de-randomized smoothing technique for the domain of malware detection. Specifically, we propose a window ablation scheme to provably limit the impact of adversarial bytes while maximally preserving local structures of the executables. After showing how DRSM is theoretically robust against attacks with contiguous adversarial bytes, we verify its performance and certified robustness experimentally, where we observe only marginal accuracy drops as the cost of robustness. To our knowledge, we are the first to offer certified robustness in the realm of static detection of malware executables. More surprisingly, through evaluating DRSM against 9 empirical attacks of different types, we observe that the proposed defense is empirically robust to some extent against a diverse set of attacks, some of which even fall out of the scope of its original threat model. In addition, we collected 15.5K recent benign raw executables from diverse sources, which will be made public as a dataset called PACE (Publicly Accessible Collection(s) of Executables) to alleviate the scarcity of publicly available benign datasets for studying malware detection and provide future research with more representative data of the time.
Features matching using natural language processing
Authors: Muhammad Danial Khilji
Date: 2023-03-14 00:00:00

The feature matching is a basic step in matching different datasets. This article proposes shows a new hybrid model of a pretrained Natural Language Processing (NLP) based model called BERT used in parallel with a statistical model based on Jaccard similarity to measure the similarity between list of features from two different datasets. This reduces the time required to search for correlations or manually match each feature from one dataset to another.
Cryptocurrency wallets: assessment and security
Authors: Ehsan Nowroozi, Seyedsadra Seyedshoari, Yassine Mekdad, Erkay Savas, Mauro Conti
Date: 2023-03-06 00:00:00

Digital wallet as a software program or a digital device allows users to conduct various transactions. Hot and cold digital wallets are considered as two types of this wallet. Digital wallets need an online connection fall into the first group, whereas digital wallets can operate without internet connection belong to the second group. Prior to buying a digital wallet, it is important to define for what purpose it will be utilized. The ease with which a mobile phone transaction may be completed in a couple of seconds and the speed with which transactions are executed are reflection of efficiency. One of the most important elements of digital wallets is data organization. Digital wallets are significantly less expensive than classic methods of transaction, which entails various charges and fees. Constantly, demand for their usage is growing due to speed, security, and the ability to conduct transactions between two users without the need of a third party. As the popularity of digital currency wallets grows, the number of security concerns impacting them increases significantly. The current status of digital wallets on the market, as well as the options for an efficient solution for obtaining and utilizing digital wallets. Finally, the digital wallets' security and future improvement prospects are discussed in this chapter.