Reviewerly/exHarmony huggingface dataset:
External knowledge document retrieval strategy based on intention-guided and meta-learning for task-oriented dialogues

The external knowledge augmented task-oriented dialogues system is designed to cover the user requests beyond the predefined data base (DB) and application-specific interface (APIs). However, the existing methods mainly focus to improve the retrieval accuracy of the external knowledge but ignore the modeling of the entity-aware dialogue intention, which can’t fast locate the document-level external API in the practical scenarios. Therefore, an external knowledge document retrieval strategy based on the intention-guided and meta-learning for the task-oriented dialogue has been proposed in this paper. First, the entity-aware dialogue intention information is introduced and then a two-stage training method for the intention-guided knowledge document retrieval model is presented, which is able to effectively enhance the retrieval accuracy of the Bi-encoder retrieval model in the task-based dialogue system. Moreover, to further improve the generalization ability of the retrieval model based on the intention-guided, a meta-task based on the domain offset is constructed and make the Bi-encoder retrieval model get the more robust semantic representation through the meta-learning, which can improve the generalization capability of the proposed model in the invisible domains. Experimental results tested on the augmented MultiWOZ 2.1 dataset are shown to prove the correctness and effectiveness of the proposed strategy.
ASDF: A Differential Testing Framework for Automatic Speech Recognition Systems

Recent years have witnessed wider adoption of Automated Speech Recognition (ASR) techniques in various domains. Consequently, evaluating and enhancing the quality of ASR systems is of great importance. This paper proposes Asdf, an Automated Speech Recognition Differential Testing Framework to test ASR systems. Asdf extends an existing ASR testing tool, the CrossASR++, which synthesizes test cases from a text corpus. However, CrossASR++ fails to make use of the text corpus efficiently and provides limited information on how the failed test cases can improve ASR systems. To address these limitations, our tool incorporates two novel features: (1) a text transformation module to boost the number of generated test cases and uncover more errors in ASR systems, and (2) a phonetic analysis module to identify phonemes that the ASR systems tend to transcribe incorrectly. Asdf generates more high-quality test cases by applying various text transformation methods (e.g., changing tense) to the input text in a failed test case. By doing so, Asdf can utilize a small text corpus to generate a large number of audio test cases, something which CrossASR++ is not capable of. In addition, Asdf implements more metrics to evaluate the performance of ASR systems from multiple perspectives. Asdf performs phonetic analysis on the identified failed test cases to identify the phonemes that ASR systems tend to transcribe incorrectly, providing useful information for developers to improve ASR systems. The demonstration video of our tool is made online at https://www.youtube.com/watch?v=DzVwfc3h9As. The implementation is available at https://github.com/danielyuenhx/asdf-differential-testing.
Hierarchical Graph Neural Network for Patient Treatment Preference Prediction with External Knowledge

The healthcare industry has a wealth of data that can be used by researchers and medical professionals to infer a patient's condition and intention to receive treatment using machine learning models. However, this line of research generally suffers from some limitations: (1) struggling to leverage structural interactions among patients; (2) attending to learn patient representations from electronic medical records (EMRs) but rarely considering supplementary contexts; and (3) overlooking EMR data imbalance issue. To address these limitations, in this paper, we propose a hierarchical graph neural network for patient treatment preference prediction. Doctors' information and their viewing activities are first integrated as external knowledge with EMRs to construct the hierarchical graph, where a dual message passing paradigm is then devised to perform intra- and inter-subgraph aggregation to enrich patient representations and advance label propagation. To mitigate patient data imbalance issue, a community detection method is further designed to better prediction. Our experimental results demonstrate the state-of-the-art performance on patient treatment preference prediction.
Diversely Regularized Matrix Factorization for Accurate and Aggregately Diversified Recommendation

When recommending personalized top-k items to users, how can we recommend them diversely while satisfying users' needs? Aggregately diversified recommender systems aim to recommend a variety of items across whole users without sacrificing the recommendation accuracy. They increase the exposure opportunities of various items, which in turn increase the potential revenue of sellers as well as user satisfaction. However, it is challenging to tackle aggregate-level diversity with matrix factorization (MF), one of the most common recommendation models, since skewed real-world data lead to the skewed recommendation results of MF. In this work, we propose DivMF (Diversely Regularized Matrix Factorization), a novel matrix factorization method for aggregately diversified recommendation. DivMF exploits novel coverage regularizer and skewness regularizer which consider the top-k recommendation results of an MF model to aggregately diversify the recommendation results. We also propose a carefully designed training algorithm for effective training. Extensive experiments on real-world datasets show that DivMF gives the state-of-the-art performance, improving up to 34.7% aggregate-level diversity in the similar level of accuracy, and up to 27.6% accuracy in the similar level of aggregate-level diversity compared to the best competitors.
Isotropic Representation Can Improve Dense Retrieval

The latest Dense Retrieval (DR) models typically encode queries and documents using BERT and subsequently apply a cosine similarity-based scoring to determine the relevance. BERT representations, however, are known to follow an anisotropic distribution of a narrow cone shape and such an anisotropic distribution can be undesirable for relevance estimation. In this work, we first show that BERT representations in DR also follow an anisotropic distribution. We adopt unsupervised post-processing methods of Normalizing Flow and whitening to cope with the problem, and develop a token-wise method in addition to the sequence-wise method. We show that the proposed methods can effectively enhance the isotropy of representations, thereby improving the performance of DR models such as ColBERT and RepBERT. To examine the potential of isotropic representation for improving the robustness of DR models, we investigate out-of-distribution tasks where the test dataset differs from the training dataset. The results show that isotropic representation can certainly achieve a generally improved performance (The code is available at https://github.com/SNU-DRL/IsotropicIR.git ).
MIDFA: Memory-Based Instance Division and Feature Aggregation Network for Video Object Detection

Previous video object detection methods focus on aggregating the features of other frames into the current frame to alleviate the image degradation, but they rarely focus on multi-class scenes. Aggregating features of different classes will generate confusing information and affect network performance. This problem can be solved by using a classifier to divide the features. However, classifier method has three problems: (a) Heterogeneous high-similarity objects and homogeneous low-similarity objects affect the accuracy of the classifier. (b) Two objects whose positions overlap also affect the classifier. (c) Previous classifier method did not exploit sufficient global information. Therefore, we propose a new method that divides the features of different instances to deal with the problems of (a) and (b). Then we designed two new memories (one is Init Memory and the other is MDR) to solve problem (c). These three parts constitute the MIDFA network. Experiments show that our method achieves 83.76% mAP on the ImageNet VID dataset based on ResNet-101, and 84.6% mAP on ResNeXt-101. In addition, we also conduct experiments on a custom-designed multi-class VID dataset, and adding Instance Division and MDR can increase the mAP of the network by 0.6% compared to using only Init Memory.
Multi-behavior Guided Temporal Graph Attention Network for Recommendation

The traditional recommendation approaches learn the representations of users and items utilizing only a single type of behavior data, which results in them facing the data sparsity issue. To alleviate the dilemma, multi-behavior recommendations leverage different types of behaviors to assist in modeling users’ preferences. Despite their remarkable effectiveness, two significant challenges have remained less explored: 1) Effectively distinguishing the contributions of different types of behaviors during capturing users’ preferences; 2) Sufficiently exploiting the temporal information of user-item interactions. To tackle these challenges, we develop a new model named Multi-behavior Guided Temporal Graph Attention Network (MB-TGAT) to discriminate the diverse influence of various behaviors and to explore the evolutionary tendencies of users’ recent preferences. In particular, we propose a behavior-aware attention mechanism to differentiate the strengths of different behaviors in the user-item aggregation phase. Furthermore, we tailor a phased message passing mechanism based on GNNs and design an evolution sequence self-attention to extract the users’ preferences from static and dynamic perspectives, respectively. Extensive experiments on three real-world datasets demonstrate the superiority of our model, noticeably with 37.27%, 37.31% and 14.63% performance gain over the state-of-the-art baselines on the Taobao, IJCAI-15 and YooChoose datasets, respectively.
SMDKGG: A Socially Aware Metadata Driven Knowledge Graph Generation for Disaster Tweets

Knowledge Graph Generation is a requisite in the modern era, where Knowledge is scarce whereas the data is exponentially surplus and extensively high. So, in this paper Knowledge Graph Generation framework, the SMDKGG has been proposed, is a Semantically Inclined Metadata driven model for Knowledge Graph Generation, which is a text to Knowledge Graph generation strategy where the document dataset is subjected to a lateral mapping of categories and alignment of generated ontologies with that of the categories for auxiliary Knowledge enhancement. Subsequently, applying Structural Topic Modeling (STM) and harvesting entities from LOD Cloud, NELL, DBPedia, and CYC Knowledge sources ensures the cohesiveness of the auxiliary Knowledge encompassed in the framework becomes extensively high, making the density of the harvested auxiliary Knowledge high. Apart from this, the Metadata generation and its classification using Transformers ensure that many more auxiliary knowledge substrates are selectively added to the framework. Its classification by a Deep Learning Transformer classifier ensures its ease of handling. Subsequently, the classification of the dataset using the AdaBoost, the encompassment of Twitter Semantic Similarity (TSS) for intra-class similarity computation, and the integration of Normalized Google Distance (NGD) with selective thresholds at various stages of the model ensure robust relevance computation mechanism for conceiving the best-in-class Knowledge Graph. Moreover, the optimization of the initial solution set is carried out using the Chemical Reaction metaheuristic optimization algorithm with NGD as the criterion function, ensuring the most relevant entities are alone conceived in the finalized Knowledge Graph. The proposed SMDKGG yields an overall Precision of 96.19%, Recall of 98.33%, Accuracy of 97.26%, F-Measure of 97.25%, and an FNR of 0.02.
A Credit Card Fraud Detection Model Using Machine Learning Methods with a Hybrid of Undersampling and Oversampling for Handling Imbalanced Datasets for High Scores

Due to recent advancements in communication technology and e-commerce systems, credit cards are currently the most popular form of payment for online as well as offline purchases. As a consequence, there is substantially more fraud involved in the transaction processes. Data mining has an issue for prediction, and data classification, therefore finding occurrences is essential. Unusual events are challenging to be identified due to their irregularity and casualness, but misclassifying unusual events can lead to large financial losses. In this study, an approach for detecting credit card fraud using machine learning methods, such as K-Nearest Neighbors, random forest, decision trees, logistic regression, and support vector machines, is proposed. The research attempts to look at the effectiveness of the classification models while applying both the Oversampling and Undersampling techniques to find instances of fraud in the dataset for fraudulent activities. The experimental study used two days of credit card transactions made by European cardholders in September 2013. To evaluate the models’ performances, confusion matrix, precision, recall, f1_score, cross-validation score, and ROC_AUC score metrics were used. From different experiments of the tested model, it can be easily observed that the performance of all models was better compared with previous literature thus the KNN was the best in almost all metrics used.
Weight Prediction Boosts the Convergence of AdamW

In this paper, we introduce weight prediction into the AdamW optimizer to boost its convergence when training the deep neural network (DNN) models. In particular, ahead of each mini-batch training, we predict the future weights according to the update rule of AdamW and then apply the predicted future weights to do both forward pass and backward propagation. In this way, the AdamW optimizer always utilizes the gradients w.r.t. the future weights instead of current weights to update the DNN parameters, making the AdamW optimizer achieve better convergence. Our proposal is simple and straightforward to implement but effective in boosting the convergence of DNN training. We performed extensive experimental evaluations on image classification and language modeling tasks to verify the effectiveness of our proposal. The experimental results validate that our proposal can boost the convergence of AdamW and achieve better accuracy than AdamW when training the DNN models.
Achieving Counterfactual Fairness for Anomaly Detection

Ensuring fairness in anomaly detection models has received much attention recently as many anomaly detection applications involve human beings. However, existing fair anomaly detection approaches mainly focus on association-based fairness notions. In this work, we target counterfactual fairness, which is a prevalent causation-based fairness notion. The goal of counterfactually fair anomaly detection is to ensure that the detection outcome of an individual in the factual world is the same as that in the counterfactual world where the individual had belonged to a different group. To this end, we propose a counterfactually fair anomaly detection (CFAD) framework which consists of two phases, counterfactual data generation and fair anomaly detection. Experimental results on a synthetic dataset and two real datasets show that CFAD can effectively detect anomalies as well as ensure counterfactual fairness.
Model-Agnostic Reachability Analysis on Deep Neural Networks

Verification plays an essential role in the formal analysis of safety-critical systems. Most current verification methods have specific requirements when working on Deep Neural Networks (DNNs). They either target one particular network category, e.g., Feedforward Neural Networks (FNNs), or networks with specific activation functions, e.g., ReLU. In this paper, we develop a model-agnostic verification framework, called DeepAgn, and show that it can be applied to FNNs, Recurrent Neural Networks (RNNs), or a mixture of both. Under the assumption of Lipschitz continuity, DeepAgn analyses the reachability of DNNs based on a novel optimisation scheme with a global convergence guarantee. It does not require access to the network’s internal structures, such as layers and parameters. Through reachability analysis, DeepAgn can tackle several well-known robustness problems, including computing the maximum safe radius for a given input, and generating the ground-truth adversarial example. We also empirically demonstrate DeepAgn’s superior capability and efficiency in handling a broader class of deep neural networks, including both FNNs and RNNs with very deep layers and millions of neurons, than other state-of-the-art verification approaches. Our tool is available at https://github.com/TrustAI/DeepAgn
F3: Fair and Federated Face Attribute Classification with Heterogeneous Data

Fairness across different demographic groups is an essential criterion for face-related tasks, Face Attribute Classification (FAC) being a prominent example. Simultaneously, federated Learning (FL) is gaining traction as a scalable paradigm for distributed training. In FL, client models trained on private datasets get aggregated by a central aggregator. Existing FL approaches require data homogeneity to ensure fairness. However, this assumption is restrictive in real-world settings. E.g., geographically distant or closely associated clients may have heterogeneous data. In this paper, we observe that existing techniques for ensuring fairness are not viable for FL with data heterogeneity. We introduce F3, an FL framework for fair FAC under data heterogeneity. We propose two methodologies in F3, (i) Heuristic-based and (ii) Gradient-based, to improve fairness across demographic groups without requiring data homogeneity assumption. We demonstrate the efficacy of our approaches through empirically observed fairness measures and accuracy guarantees on popular face datasets. Using Mahalanobis distance, we show that F3 obtains a practical balance between accuracy and fairness for FAC. The code is available at: github.com/magnetar-iiith/F3 .
Small Temperature is All You Need for Differentiable Architecture Search

Differentiable architecture search (DARTS) yields highly efficient gradient-based neural architecture search (NAS) by relaxing the discrete operation selection to optimize continuous architecture parameters that maps NAS from the discrete optimization to a continuous problem. DARTS then remaps the relaxed supernet back to the discrete space by one-off post-search pruning to obtain the final architecture (finalnet). Some emerging works argue that this remap is inherently prone to mismatch the network between training and evaluation which leads to performance discrepancy and even model collapse in extreme cases. We propose to close the gap between the relaxed supernet in training and the pruned finalnet in evaluation through utilizing small temperature to sparsify the continuous distribution in the training phase. To this end, we first formulate sparse-noisy softmax to get around gradient saturation. We then propose an exponential temperature schedule to better control the outbound distribution and elaborate an entropy-based adaptive scheme to finally achieve the enhancement. We conduct extensive experiments to verify the efficiency and efficacy of our method.
Assessment of Lung Cancer Histology Using Efficient Net

Lung cancer patients must be diagnosed early in order to have a better chance of survival. Tissue histopathology is a common method for early diagnosis. Typically, a pathologist is in charge of tissue analysis, which is a prolonged and error-prone procedure. If cancer zones could be detected, the process would be greatly accelerated, and the pathologist would be greatly aided. In this study, the authors designed and implemented an automated method for detecting lung cancer throughout the entire slide. The authors used the Efficientnet family, Efficientnet-B0 to Efficientnet-B3, to classify histopathological images into one of three classes. The Efficientnet is the most powerful CNN, which optimizes both accuracy and efficiency. The models are compared in terms of accuracy and training time. The findings demonstrate that Efficientnet-B2, which obtained the highest validation accuracy of 95.84%, is a trustworthy model. Further, it can be stated that Efficientnet-B3 is over-parameterized for the small dataset of lung cancer and does not improve the model accuracy.
Explainability of Machine Learning in Work Disability Risk Prediction

The risk of work disability can be predicted with machine learning (ML). All stakeholders should understand how the estimation is done to trust the results. Thus, these methods should be not only sufficiently accurate but also transparent and explainable. Explainability is one topic in artificial intelligence (AI) Ethics. Explainable AI (XAI) is especially important in health-related topics. We compared the accuracy of two ML methods that use occupational health care data (MHealth) and pension decision register data (MPension). Method MHealth uses deep neural networks and natural language processing (NLP) algorithms and method MPension uses different decision tree algorithms. We can assume that both methods are black box predictors because the reasoning behind the function is not understandable by humans. We observed in our previous study that both methods are sufficiently accurate to support experts in decision making. Our aim in this study was to determine if these methods are sufficiently explainable for clinical use. The two main approaches to estimating the explainability of the ML methods are transparency design and post-hoc explanations. We could not access the data for these methods and had to limit our research to the post-hoc explanation approach. We formulated the visualizations for the methods MHealth and MPension and discussed how understandable they are. We also determined that the explainability is better in MPension but the deep learning algorithm in MHealth is also possible to visualize.
Natural Language Processing For Automatic text summarization [Datasets] - Survey

Natural language processing has developed significantly recently, which has progressed the text summarization task. It is no longer limited to reducing the text size or obtaining helpful information from a long document only. It has begun to be used in getting answers from summarization, measuring the quality of sentiment analysis systems, research and mining techniques, document categorization, and natural language Inference, which increased the importance of scientific research to get a good summary. This paper reviews the most used datasets in text summarization in different languages and types, with the most effective methods for each dataset. The results are shown using text summarization matrices. The review indicates that the pre-training models achieved the highest results in the summary measures in most of the researchers' works for the datasets. Dataset English made up about 75% of the databases available to researchers due to the extensive use of the English language. Other languages such as Arabic, Hindi, and others suffered from low resources of dataset sources, which limited progress in the academic field.