Reviewerly/exHarmony huggingface dataset:
Adversarial Examples Detection Based on Error Level Analysis and Space Mapping

Deep neural network (DNN) shows impressive performance on many tasks but they usually suffer from adversarial examples with human eyes invisible slight perturbation. Such examples can not be distinguished by human but can mislead DNN classifiers leading to its important role in DNN attack and defense. Many adversarial examples detection methods perform well in identifying global perturbation adversarial examples but less efficiently for local perturbation ones. We observe both global perturbation and local perturbation adversarial examples have similar BOF histogram distribution after JPEG compression and Error Level Analysis (ELA) while these distributions are clearly different to clean example’s distribution. Meanwhile, researchers have found that the stability of adversarial example after space mapping is worse than that of the clean example. Therefore, we propose a two-branch architecture to detect adversarial examples based on the aforementioned strategies. Experiments show that our method has achieved better or similar performance compared to several state-of-the-art methods in terms of the detection accuracy and generation property for adversarial examples with global and local perturbation.
Win The Lottery Ticket Via Fourier Analysis: Frequencies Guided Network Pruning

With the remarkable success of deep learning recently, efficient network compression algorithms are urgently demanded for releasing the potential computational power of edge devices, such as smartphones or tablets. However, optimal network pruning is a non-trivial task which mathematically is an NP-hard problem. Previous researchers explain training a pruned network as buying a lottery ticket. In this paper, we investigate the Magnitude-Based Pruning (MBP) scheme and analyze it from a novel perspective through Fourier analysis on the deep learning model to guide model designation. Besides explaining the generalization ability of MBP using Fourier transform, we also propose a novel two-stage pruning approach, where one stage is to obtain the topological structure of the pruned network and the other stage is to retrain the pruned network to recover the capacity using knowledge distillation from lower to higher on the frequency domain. Extensive experiments on CIFAR-10 and CIFAR-100 demonstrate the superiority of our novel Fourier analysis based MBP compared to other traditional MBP algorithms.
Sparsity Improves Unsupervised Attribute Discovery in Stylegan

Rich semantics exist in latent spaces inferred using deep generative models. The ability to extract and interpret them is not only essential for understanding the underlying factors of variation in the data distribution, but also crucial for con-trolled image generation. Several methods have been proposed to identify semantically meaningful linear directions, either through existing annotations, or relying on identifying directions of large variation that arise from the data representation of the network. In this paper, we identify a new criterion, representation sparsity, that allows us to produce extremely efficient yet diverse semantic directions in GAN (generative adversarial network) latent spaces. The observation also reveals a potential deeper connection between representation sparsity and semantics in deep neural networks that worth further exploration.
Relevance Models Based on the Knowledge Gap

Search systems are increasingly used for gaining knowledge through accessing relevant resources from a vast volume of content. However, search systems provide only limited support to users in knowledge acquisition contexts. Specifically, they do not fully consider the knowledge gap which we define as the gap existing between what the user knows and what the user intends to learn. The effects of considering the knowledge gap for knowledge acquisition tasks remain largely unexplored in search systems. We propose to model and incorporate the knowledge gap into search algorithms. We plan to explore to what extent the incorporation of the knowledge gap leads to an improvement in the performance of search systems in knowledge acquisition tasks. Furthermore, we aim to investigate and design a metric for the evaluation of the search systems' performance in the context of knowledge acquisition tasks.
Deployment of Medibot in Medical Field

During this pandemic situation most of the people's health are in the need of medicine and doctors suggestions to improve and protect their health. Also, have seen many such cases where many people have been infected by COVID. To reduce the physical contact and help the people from the spread of diseases the proposed methodology is to implement the medibot in hospitals. A medical bot is a Chatbot which uses NLP (Natural Language Processing) by text-format. The medibot is supported by AI and Deep Learning for Medical Diagnostics. The goal of the project is to create a medibot that overcomes the proposed methodology. There were many people who could not meet the doctors for simple problems such as cold and fever. To reduce these cases will implement the medibot. This medibot can communicate with the patients and understand the symptoms, it will also give them medicines.
Construction of Psychological Counseling Intelligent Platform based on Complexity Test Algorithm

By analyzing the target image of the complexity test algorithm, the quantitative description of the psychological state of college students is extracted, and an intelligent platform for psychological counseling is established. Data variance weighted information entropy, gradient direction features, and local contrast features. Using the method of multi-feature fusion, the psychological variance weighted information entropy feature, gradient direction feature and local contrast feature are defined as the complexity feature of the image, and then the effective feature extraction and segmentation of college students' psychological counseling is realized, and the target detection is completed.
Bridging the gap between one-to-many and one-to-one label assignment via NMS-aware alignment module

The recently proposed one-to-one label assignment rules have made it possible for the detector to get rid of its dependence on NMS. However, its performance is inferior to conventional one-to-many label assignment trained detector due to insufficient supervision caused by reduced foreground information. In this paper, to combine the advantages of these two kinds of label assignment rules, we assign two sets of positive samples by one-to-many and one-to-one label assignment rules respectively to train the detector. Furthermore, we introduce the graph convolutional network (GCN) to construct a NMS-aware alignment module. The NMS-aware alignment module is devised to bridge the gap between the two sets of positive samples and generate a mask to select out best detections, which plays the role of NMS. It also gets rid of the limitation of processing the grid-like data structure and effectively aggregates information from the duplicate predictions. Extensive experiments on the COCO dataset show the effectiveness of our method and the proposed method achieves competitive performance against the recent NMS-free detectors. Especially, our proposed method is compatible with both anchor-based and anchor-free or one-stage and two-stage architectures. The code will be made public soon.
Hybrid Algorithm for Security of Data Over Wireless Transfer with Disease Prediction in Machine Learning

Data security is major issue in the entire field. With the evolving technological advancements in our day to day life security of personal data is a primary concern for all. Many intruders and hackers are eagerly waiting for personal data of the people which can be used in various means for their works. They always have an eye on the personal information of the people. And the second issue is that diagnosing of diabetes by the doctor will be some time prone to mistakes because it is natural that human makes mistakes but not the machines. So to overcome this issue we develop a hybrid secure algorithm which encrypts the patient personal information which they wish not to share to anyone outside the hospital by crypto AES encryption and hashing using SHA-256 and then storing the data in database which will be accessible only to the patient and the doctor. Together with encryption of the patient personal data and medical data our proposed system will effectively evaluate the data by a trained machine learning model to check whether a person is actually suffering from diabetes or not. We use logistic regression for evaluating the patient medical data to predict the health status so mistakes can be avoided in prediction as the prediction is done by the trained machine. Then doctor can give the treatment depending on the health status. As a result this system will give a sustainable solution to people for safely transmitting their personal data as well as medical data through wireless medium along with diabetes disease prediction.
Novel Class Discovery: A Dependency Approach

Supervised and semi-supervised algorithms have been designed under a closed-world setting, with the assumption that unlabeled data consists of classes previously seen in labeled training data. However, real world is inherently open set where this assumption is often violated, and thus novel data may be encountered in test data. In this paper, we look at the problem where the model is required to discover novel classes never encountered in the labeled set. We propose a dependency measure based on Squared Mutual Information (SMI) where we simultaneously learn to classify and cluster the data. Our experiments show that our approach is able to achieve competitive performance on CIFAR and Imagenet datasets.
Robust Unstructured Knowledge Access in Conversational Dialogue with ASR Errors

Performance of spoken language understanding (SLU) can be degraded with automatic speech recognition (ASR) errors. We propose a novel approach to improve SLU robustness by randomly corrupting clean training text with an ASR error simulator, followed by self-correcting the errors and minimizing the target classification loss in a joint manner. In the proposed error simulator, we leverage confusion networks generated from an ASR decoder without human transcriptions to generate a variety of error patterns for model training. We evaluate our approach on the DSTC10 challenge targeted for knowledge-grounded task-oriented conversational dialogues with ASR errors. Experimental results show the effectiveness of our proposed approach, boosting the knowledge-seeking turn detection (KTD) F1 significantly from 0.9433 to 0.9904. Knowledge cluster classification is boosted from 0.7924 to 0.9333 in Recall@1. After knowledge document re-ranking, our approach shows significant improvement in all knowledge selection metrics, from 0.7358 to 0.7806 in Recall@1, from 0.8301 to 0.9333 in Recall@5, and from 0.7798 to 0.8460 in MRR@5 on the test set. In the recent DSTC10 evaluation, our approach demonstrates significant improvement in knowledge selection, boosting Recall@1 from 0.495 to 0.7144 compared to the official baseline. Our source code is released in GitHub https://github.com/yctam/dstc10_track2_task2.git.
Improving Anomaly Detection with a Self-Supervised Task Based on Generative Adversarial Network

Existing anomaly detection models show success in detecting abnormal images with generative adversarial networks on the insufficient annotation of anomalous samples. However, existing models cannot accurately identify the anomaly samples which are close to the normal samples. We assume that the main reason is that these methods ignore the diversity of patterns in normal samples. To alleviate the above issue, this paper proposes a novel anomaly detection framework based on generative adversarial network, called ADe-GAN. More concretely, we construct a self-supervised learning task to fully explore the pattern information and latent representations of input images. In model inferring stage, we design a new abnormality score approach by jointly considering the pattern information and reconstruction errors to improve the performance of anomaly detection. Extensive experiments show that the ADe-GAN outperforms the state-of-the-art methods over several real-world datasets.
Mimo Detection by Variational Posterior Inference

In this paper we examine the application of variational inference (VI) to MIMO detection. Our study is motivated by the recent interest in applying machine learning concepts to signal processing. VI is an approach for providing friendly approximations of certain intractable posterior probabilities in statistics, and it has been popularly used in machine learning. In MIMO detection we also have a similar problem; specifically, we want to evaluate the posterior symbol probabilities for detection, but they are computationally too expensive to evaluate when the problem size and/or the constellation size are large. By approximating the discrete symbol prior by a continuous Gaussian mixture model, we show how the notion of VI can be used to derive an iterative MIMO detector. Interestingly, the detector resembles the MMSE detector in structure. The performance of the proposed detector is demonstrated by simulations.
BNU: A Balance-Normalization-Uncertainty Model for Incremental Event Detection

Event detection is challenging in real-world application since new events continually occur and old events still exist which may result in repeated labeling for old events. Therefore, incremental event detection is essential where a model continuously learns new events and meanwhile prevents performance from degrading on old events. Although existing incremental event detection models achieve impressive performance, they face the data imbalance problem between old classes and new classes, and have the knowledge transfer problem which cannot adequately utilize the knowledge provided by the previous model and data. To this end, we propose a Balance-Normalization-Uncertainty (BNU) model to address above problems. Specifically, in order to mitigate the adverse effects of data imbalance, we incorporate a balanced fine-tuning stage and a cosine normalization module. Meanwhile, we consider aleatoric uncertainty to preserve previous knowledge while training for new events. Experimental results show that our proposed method resolves the above challenges effectively and achieves consistent and significant performance on ACE and TAC KBP datasets.
Effective and Inconspicuous Over-the-Air Adversarial Examples with Adaptive Filtering

While deep neural networks achieve state-of-the-art performance on many audio classification tasks, they are known to be vulnerable to adversarial examples - artificially-generated perturbations of natural instances that cause a network to make incorrect predictions. In this work we demonstrate a novel audio-domain adversarial attack that modifies benign audio using an interpretable and differentiable parametric transformation - adaptive filtering. Unlike existing state-of-the-art attacks, our proposed method does not require a complex optimization procedure or generative model, relying only on a simple variant of gradient descent to tune filter parameters. We demonstrate the effectiveness of our method by performing over-the-air attacks against a state-of-the-art speaker verification model and show that our attack is less conspicuous than an existing state-of-the-art attack while matching its effectiveness. Our results demonstrate the potential of transformations beyond direct waveform addition for concealing high-magnitude adversarial perturbations, allowing adversaries to attack more effectively in challenging, real-world settings.
Multi-Turn Incomplete Utterance Restoration As Object Detection

In this paper, we investigate the task of multi-turn incomplete utterance restoration to tackle the issue of frequent coreference and information omission in multi-turn dialogues. Recent works mainly focus on edit-based approaches which have been proven to outperform traditional generation-based models in terms of accuracy and efficiency. However, they only model token-level edit relationships while ignoring span-level edit relationships. Our experiments find this breaks the semantic integrity of edit span, which causes inaccurate edit span prediction and disfluent utterance restoration. To address the problem, we propose a novel approach to directly model span-level edit relationships between the incomplete utterance and context. Specifically, we build an edit matrix in which each rectangular region represents a span-level edit operation. Then, we detect the region with a well-designed dual-branch detection module inspired by object detection. Empirical results demonstrate that our method outperforms state-of-the-art methods significantly on two public datasets. In addition, further studies verify that our method is capable of preserving the semantic integrity of edit span.
MAKD:MULTIPLE Auxiliary Knowledge Distillation

Knowledge distillation aims to learn a small student model by leveraging knowledge from a larger teacher model. The gap between these heterogeneous models hinder their knowledge transfer and it would be more challenging when the teacher model is from another task. Previous methods view the teacher model as a perfect feature extractor and train the student model to mimic it. While we notice that the teacher model has defect in extracting features of another task samples. To improve knowledge distillation under such situation, we propose Multiple Auxiliary Subspaces (MAS). Most previous methods improve distillation performance by representation alignment, while we resort to the promotion of the teacher model which is more suitable for cross-task distillation. The MAS distills the knowledge in a mutual learning way based on an auxiliary network. Along with the training procedure, the teacher model is improved by the auxiliary network which works as a trainable part of the teacher model and learn the features distribution of target samples from the student model. And this promotion of the teacher model will benefit the student model via the following distillation procedure. We adopt the representation alignment technique, multiple auxiliary networks to further enhance the proposed method. The MAS works well with limited or sufficient labeled target data. If the source data is available, based on it the MAS can construct another auxiliary network to further improve the distillation. Experiments are conducted to validate that the MAS outperforms baseline methods and achieves state-of-the-art results on several standard benchmarks.
PGTRNET: Two-Phase Weakly Supervised Object Detection with Pseudo Ground Truth Refinement

Current state-of-the-art weakly supervised object detection (WSOD) studies mainly follow a two-stage training strategy which integrates a fully supervised detector (FSD) with a pure WSOD model. There are two main problems hindering the performance of the two-phase WSOD approaches, i.e., insufficient learning problem and strict reliance between the FSD and the pseudo ground truth (PGT) generated by the WSOD model. This paper proposes pseudo ground truth refinement network (PGTRNet), a simple yet effective method with-out introducing any extra learnable parameters, to cope with these problems. PGTRNet utilizes multiple bounding boxes to establish the PGT, mitigating the insufficient learning problem. Besides, we propose a novel online PGT refinement approach to steadily improve the quality of PGT by fully taking advantage of the power of FSD during the second-phase training, decoupling the first and second-phase models. Elaborate experiments are conducted on the PASCAL VOC 2007 benchmark to verify the effectiveness of our methods. Experimental results demonstrate that PGTRNet boosts the backbone model by 2.1% mAP and achieves the state-of-the-art performance.
Being Greedy Does Not Hurt: Sampling Strategies for End-To-End Speech Recognition

Maximum Likelihood Estimation (MLE) is currently the most common approach to train large scale speech recognition systems. While it has significant practical advantages, MLE exhibits several drawbacks known in literature: training and inference conditions are mismatched and a proxy objective is optimized instead of word error rate. Recently, the Optimal Completion Distillation (OCD) training method was proposed which attempts to address some of those issues. In this paper, we analyze if the method is competitive over a strong MLE baseline and investigate its scalability towards large speech data beyond read speech, which to our knowledge is the first attempt known in literature. In addition, we propose and analyze several sampling strategies trading off exploration and exploitation of unseen prefixes and their effect on ASR accuracy. We conduct several experiments on both public LibriSpeech data and in-house large scale far-field data and compare models trained with MLE and OCD. Our proposed greedy sampling with soft targets approach proves most effective and yields a 9% rel. word error rate improvement over the i.i.d sampling. Finally, we note that OCD method improves over the MLE without label smoothing by 12%, and underperform by 6% once label smoothing is introduced to MLE.
Exploring Category Consistency for Weakly Supervised Semantic Segmentation

Self-supervised framework has been widely used in weakly supervised semantic segmentation. Generating a reliable and detailed pseudo mask label is the main challenge for improving the quality of predicted mask. In this paper, we propose Category Consistency Mask Refinement (CCMR) to explore the category consistency cued with the input image, and inject such information to mask refinement, guaranteeing the completeness of the refined mask. Moreover, we exploit Selective Weighted Pooling (SWP) to restrict the backward propagation of background, limiting the update of the background. Experimental results demonstrate that our methods can boost the performance on the PASCAL VOC 2012 segmentation benchmark, outperforming the state-of-the-art weakly supervised semantic segmentation methods.
Wasserstein Cross-Lingual Alignment For Named Entity Recognition

Supervised training of Named Entity Recognition (NER) models generally require large amounts of annotations, which are hardly available for less widely used (low resource) languages, e.g., Armenian and Dutch. Therefore, it will be desirable if we could leverage knowledge extracted from a high resource language (source), e.g., English, so that NER models for the low resource languages (target) could be trained more efficiently with less cost associated with annotations. In this paper, we study cross-lingual alignment for NER, an approach for transferring knowledge from high-to low-resource languages, via the alignment of token embeddings between different languages. Specifically, we propose to align by minimizing the Wasserstein distance between the contextualized token embeddings from source and target languages. Experimental results show that our method yields improved performance over existing works for cross-lingual alignment in NER tasks.