Reviewerly/exHarmony huggingface dataset:
Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning

Pre-Trained Models have been widely applied and recently proved vulnerable under backdoor attacks: the released pre-trained weights can be maliciously poisoned with certain triggers.When the triggers are activated, even the fine-tuned model will predict pre-defined labels, causing a security threat.These backdoors generated by the poisoning methods can be erased by changing hyper-parameters during fine-tuning or detected by finding the triggers.In this paper, we propose a stronger weight-poisoning attack method that introduces a layerwise weight poisoning strategy to plant deeper backdoors; we also introduce a combinatorial trigger that cannot be easily detected.The experiments on text classification tasks show that previous defense methods cannot resist our weight-poisoning method, which indicates that our method can be widely applied and may provide hints for future model robustness studies.
Low-Dose CT Image Denoising with Improving WGAN and Hybrid Loss Function

The X-ray radiation from computed tomography (CT) brought us the potential risk. Simply decreasing the dose makes the CT images noisy and diagnostic performance compromised. Here, we develop a novel denoising low-dose CT image method. Our framework is based on an improved generative adversarial network coupling with the hybrid loss function, including the adversarial loss, perceptual loss, sharpness loss, and structural similarity loss. Among the loss function terms, perceptual loss and structural similarity loss are made use of to preserve textural details, and sharpness loss can make reconstruction images clear. The adversarial loss can sharp the boundary regions. The results of experiments show the proposed method can effectively remove noise and artifacts better than the state-of-the-art methods in the aspects of the visual effect, the quantitative measurements, and the texture details.
Progress toward favorable landscapes in quantum combinatorial optimization

The performance of variational quantum algorithms relies on the success of using quantum and classical computing resources in tandem. Here, we study how these quantum and classical components interrelate. In particular, we focus on algorithms for solving the combinatorial optimization problem MaxCut, and study how the structure of the classical optimization landscape relates to the quantum circuit used to evaluate the MaxCut objective function. In order to analytically characterize the impact of quantum features on the critical points of the landscape, we consider a family of quantum circuit ans\"atze composed of mutually commuting elements. We identify multiqubit operations as a key resource and show that overparameterization allows for obtaining favorable landscapes. Namely, we prove that an ansatz from this family containing exponentially many variational parameters yields a landscape free of local optima for generic graphs. However, we further prove that these ans\"atze do not offer superpolynomial advantages over purely classical MaxCut algorithms. We then present a series of numerical experiments illustrating that noncommutativity and entanglement are important features for improving algorithm performance.
Statistical Mechanics of Deep Linear Neural Networks: The Backpropagating Kernel Renormalization

The groundbreaking success of deep learning in many real-world tasks has triggered an intense effort to theoretically understand the power and limitations of deep learning in the training and generalization of complex tasks, so far with limited progress. In this work, we study the statistical mechanics of learning in deep linear neural networks (DLNNs) in which the input-output function of an individual unit is linear. Despite the linearity of the units, learning in DLNNs is highly nonlinear; hence, studying its properties reveals some of the essential features of nonlinear deep neural networks (DNNs). Importantly, we exactly solve the network properties following supervised learning using an equilibrium Gibbs distribution in the weight space. To do this, we introduce the backpropagating kernel renormalization (BPKR), which allows for the incremental integration of the network weights layer by layer starting from the network output layer and progressing backward until the first layer's weights are integrated out. This procedure allows us to evaluate important network properties, such as its generalization error, the role of network width and depth, the impact of the size of the training set, and the effects of weight regularization and learning stochasticity. BPKR does not assume specific statistics of the input or the task's output. Furthermore, by performing partial integration of the layers, the BPKR allows us to compute the emergent properties of the neural representations across the different hidden layers. We propose a heuristic extension of the BPKR to nonlinear DNNs with rectified linear units (ReLU). Surprisingly, our numerical simulations reveal that despite the nonlinearity, the predictions of our theory are largely shared by ReLU networks of modest depth, in a wide regime of parameters. Our work is the first exact statistical mechanical study of learning in a family of deep neural networks, and the first successful theory of learning through the successive integration of degrees of freedom in the learned weight space.8 MoreReceived 6 December 2020Revised 27 May 2021Accepted 9 July 2021DOI:https://doi.org/10.1103/PhysRevX.11.031059Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasNeurosciencePhysical SystemsArtificial neural networksNeuronal networksTechniquesLangevin algorithmLangevin equationRenormalization groupReplica methodsBiological PhysicsNetworksStatistical PhysicsInterdisciplinary Physics
Toward Accurate Pixelwise Object Tracking via Attention Retrieval

Pixelwise single object tracking is challenging due to the competition of running speeds and segmentation accuracy. Current state-of-the-art real-time approaches seamlessly connect tracking and segmentation by sharing computation of the backbone network, e.g., SiamMask and D3S fork a light branch from the tracking model to predict segmentation mask. Although efficient, directly reusing features from tracking networks may harm the segmentation accuracy, since background clutter in the backbone feature tends to introduce false positives in segmentation. To mitigate this problem, we propose a unified tracking-retrieval-segmentation framework consisting of an attention retrieval network (ARN) and an iterative feedback network (IFN). Instead of segmenting the target inside the bounding box, the proposed framework performs soft spatial constraints on backbone features to obtain an accurate global segmentation map. Concretely, in ARN, a look-up-table (LUT) is first built by sufficiently using the information of the first frame. By retrieving it, a target-aware attention map is generated to suppress the negative influence of background clutter. To ulteriorly refine the contour of the segmentation, IFN iteratively enhances the features at different resolutions by taking the predicted mask as feedback guidance. Our framework sets a new state of the art on the recent pixelwise tracking benchmark VOT2020 and runs at 40 fps. Notably, the proposed model surpasses SiamMask by 11.7/4.2/5.5 points on VOT2020, DAVIS2016, and DAVIS2017, respectively. Code is available at https://github.com/JudasDie/SOTS.
Few-Shot Object Detection on Remote Sensing Images via Shared Attention Module and Balanced Fine-Tuning Strategy

Few-shot object detection is a recently emerging branch in the field of computer vision. Recent research studies have proposed several effective methods for object detection with few samples. However, their performances are limited when applied to remote sensing images. In this article, we specifically analyze the characteristics of remote sensing images and propose a few-shot fine-tuning network with a shared attention module (SAM) to adapt to detecting remote sensing objects, which have large size variations. In our SAM, multi-attention maps are computed in the base training stage and shared with the feature extractor in the few-shot fine-tuning stage as prior knowledge to help better locate novel class objects with few samples. Moreover, we design a new few-shot fine-tuning stage with a balanced fine-tuning strategy (BFS), which helps in mitigating the severe imbalance between the number of novel class samples and base class samples caused by the few-shot settings to improve the classification accuracy. We have conducted experiments on two remote sensing datasets (NWPU VHR-10 and DIOR), and the excellent results demonstrate that our method makes full use of the advantages of few-shot learning and the characteristics of remote sensing images to enhance the few-shot detection performance.
Securing Genetic Algorithm Enabled SDN Routing for Blockchain Based Internet of Things

Internet of Things (IoT) is an emerging domain in which different devices communicate with each other through minimum human intervention. IoT devices are usually operated in hostile and unattended environments. Moreover, routing in current IoT architecture becomes inefficient due to malicious and unauthenticated nodes’ existence, minimum network lifetime, insecure routing, etc. This paper proposes a lightweight blockchain based authentication mechanism where ordinary sensors’ credentials are stored. As IoT nodes have a short lifespan due to energy depletion, few credentials are stored in the blockchain to achieve lightweight authentication. Moreover, the route calculation is performed by a genetic algorithm enabled software defined network controller, which is also used for on-demand routing to optimize the energy consumption of the nodes in the IoT network. Furthermore, a route correctness mechanism is proposed to check the existence of malicious nodes in the calculated route. Moreover, a detection mechanism is proposed to restrict the malicious nodes’ activities, while a malicious node’s list is maintained in the blockchain, which is used in the route correctness mechanism. The proposed model is evaluated by performing intensive simulations. The effectiveness of the proposed model is depicted in terms of gas consumption, which shows the optimized utilization of resources. The residual energy of the network shows optimized route calculation, while the malicious node detection method shows the number of packets dropped.
ALOA: Ant Lion Optimization Algorithm-based Deep Learning for Breast Cancer Classification

Normally, breast cancer is the most familiar disease, and it is the second leading type of cancer death among women.Breast cancer disease begins in malignant stage and it slowly develops from breast cell.Periodic clinical checks and self-tests assist for early prediction of breast cancer, thus the survival rate can increase significantly.In this paper, Ant Lion Optimization (ALO)-based Deep Neural Network (DNN) technique is developed for breast cancer classification.Here, the Type 2 Fuzzy and Cuckoo Search (T2FCS) method is utilized for pre-processing the input image.The color-based thresholding scheme is employed for segmenting the blood cells from the input breast cancer image.Furthermore, the statistical features are extracted from an input image for predicting breast cancer.Besides, the cancer classification is performed using a DNN classifier.The ALO algorithm is developed in this method to train the classifier for better performance.In addition, the developed breast cancer classification approach achieves a better classification performance based on several parameters, such as specificity, accuracy, and sensitivity.Thus, the developed ALO-based DNN method obtained enhanced results with high accuracy of 95.34%, sensitivity of 97.54%, and specificity of 95.34%.
Circle Representation for Medical Object Detection

Box representation has been extensively used for object detection in computer vision. Such representation is efficacious but not necessarily optimized for biomedical objects (e.g., glomeruli), which play an essential role in renal pathology. In this paper, we propose a simple circle representation for medical object detection and introduce CircleNet, an anchor-free detection framework. Compared with the conventional bounding box representation, the proposed bounding circle representation innovates in three-fold: (1) it is optimized for ball-shaped biomedical objects; (2) The circle representation reduced the degree of freedom compared with box representation; (3) It is naturally more rotation invariant. When detecting glomeruli and nuclei on pathological images, the proposed circle representation achieved superior detection performance and be more rotation-invariant, compared with the bounding box. The code has been made publicly available: <uri>https://github.com/hrlblab/CircleNet</uri>.
Attention-based end-to-end CNN framework for content-based X-ray image retrieval

The widespread use of medical imaging devices allows deep analysis of diseases.However, the task of examining medical images increases the burden of specialist doctors.Computer-assisted systems provide an effective management tool that enables these images to be analyzed automatically.Although these tools are used for various purposes, today, they are moving towards retrieval systems to access increasing data quickly.In hospitals, the need for content-based image retrieval systems is seriously evident in order to store all images effectively and access them quickly when necessary.In this study, an attention-based end-to-end convolutional neural network (CNN)framework that can provide effective access to similar images from a large X-ray dataset is presented.In the first part of the proposed framework, a fully convolutional network architecture with attention structures is presented.This section contains several layers for determining the saliency points of X-ray images.In the second part of the framework, the modified image with X-ray saliency map is converted to representative codes in Euclidean space by the ResNet-18 architecture.Finally, hash codes are obtained by transforming these codes into hamming spaces.The proposed study is superior in terms of high performance and customized layers compared to current state-of-the-art X-ray image retrieval methods in the literature.Extensive experimental studies reveal that the proposed framework can increase the current precision performance by up to 13
A Quadruplet Deep Metric Learning model for imbalanced time-series fault diagnosis

Deep learning based methods are attractive and meaningful in the field of fault diagnosis recently. However, the sample size of different faults may be imbalanced in practical application scenarios, which results in performance degeneration. The optimization of data representation could be an effective way to alleviate such phenomenon. In this paper, a deep learning framework considering time series and distance metric called LSTM-Quadruplet Deep Metric Learning(LSTM-QDM) model is proposed. It maps original space to a feature space where the distribution of imbalance faults is more distinguishable. A novel quadruplet data pair is designed which adds a minor sample from imbalanced classes into traditional data pair. Based on such data pair, a quadruplet loss function is proposed to increase the distance between the imbalanced classes and other classes, and its combination with softmax loss would improve the representation ability and classification performance simultaneously. The proposed data pair and loss function encourage the full mining of imbalanced data in the training process. The experiments are carried out on two open-source datasets including TE and CWRU, and the fault diagnosis performance is validated under different imbalanced conditions. The experimental results indicate that our proposed model is effective and robust in the imbalanced fault diagnosis task.
OOD-GNN: Out-of-Distribution Generalized Graph Neural Network

Graph neural networks (GNNs) have achieved impressive performance when testing and training graph data come from identical distribution. However, existing GNNs lack out-of-distribution generalization abilities so that their performance substantially degrades when there exist distribution shifts between testing and training graph data. To solve this problem, in this work, we propose an out-of-distribution generalized graph neural network (<b>OOD-GNN</b>) for achieving satisfactory performance on unseen testing graphs that have different distributions with training graphs. Our proposed <b>OOD-GNN</b> employs a novel nonlinear graph representation decorrelation method utilizing random Fourier features, which encourages the model to eliminate the statistical dependence between relevant and irrelevant graph representations through iteratively optimizing the sample graph weights and graph encoder. We further present a global weight estimator to learn weights for training graphs such that variables in graph representations are forced to be independent. The learned weights help the graph encoder to get rid of spurious correlations and, in turn, concentrate more on the true connection between learned discriminative graph representations and their ground-truth labels. We conduct extensive experiments to validate the out-of-distribution generalization abilities on two synthetic and 12 real-world datasets with distribution shifts. The results demonstrate that our proposed <b>OOD-GNN</b> significantly outperforms state-of-the-art baselines.
Entity-Based Knowledge Conflicts in Question Answering

Knowledge-dependent tasks typically use two sources of knowledge: parametric, learned at training time, and contextual, given as a passage at inference time.To understand how models use these sources together, we formalize the problem of knowledge conflicts, where the contextual information contradicts the learned information.Analyzing the behaviour of popular models, we measure their over-reliance on memorized information (the cause of hallucinations), and uncover important factors that exacerbate this behaviour.Lastly, we propose a simple method to mitigate over-reliance on parametric knowledge which minimizes hallucination and improves out-of-distribution generalization by 4% -7%.Our findings demonstrate the importance for practitioners to evaluate model tendency to hallucinate rather than read, and show that our mitigation strategy encourages generalization to evolving information (i.e., time-dependent queries).To encourage these practices, we have released our framework for generating knowledge conflicts.
Examination of Teachers’ Digital Literacy Levels and Life Long Learning Tendencies

In this study, digital literacy levels and lifelong learning tendencies of teachers were investigated. Data were collected from 210 teachers working in public schools in Zonguldak province. Two measurement tools, namely Digital Literacy Scale and Lifelong Learning Tendency Scale, were used in data collection. In the study conducted with the scanning method, the difference between the variables of gender, branch and age of the teachers and digital literacy and lifelong learning tendencies was investigated. In addition, the relationship between teachers’ digital literacy levels and lifelong learning tendencies was examined. Descriptive statistics, independent samples t-test and one-way analysis of variance (ANOVA) were used in the analysis of the data. According to the results obtained from the research, no significant difference was found between the digital literacy levels and lifelong learning tendencies of teachers and gender, educational status (undergraduate-graduate), branch (Classroom teacher-Culture lesson teacher). It was observed that the digital literacy levels of teachers aged 46 and over were lower than those of younger teachers. No significant difference was found between the ages of teachers and their lifelong learning tendencies. It has been observed that there is a medium-level positive relationship between teachers’ lifelong learning tendencies and digital literacy levels.
A Systematic Review of Machine Learning Algorithms in Cyberbullying Detection: Future Directions and Challenges

Social media networks are becoming an essential part of life for most of the world’s population. Detecting cyberbullying using machine learning and natural language processing algorithms is getting the attention of researchers. There is a growing need for automatic detection and mitigation of cyberbullying events on social media. In this study, research directions and the theoretical foundation in this area are investigated. A systematic review of the current state-of-the-art research in this area is conducted. A framework considering all possible actors in the cyberbullying event must be designed, including various aspects of cyberbullying and its effect on the participating actors. Furthermore, future directions and challenges are also discussed.
Strongly nonlocal unextendible product bases do exist

A set of multipartite orthogonal product states is locally irreducible, if it is not possible to eliminate one or more states from the set by orthogonality-preserving local measurements. An effective way to prove that a set is locally irreducible is to show that only trivial orthogonality-preserving local measurement can be performed to this set. In general, it is difficult to show that such an orthogonality-preserving local measurement must be trivial. In this work, we develop two basic techniques to deal with this problem. Using these techniques, we successfully show the existence of unextendible product bases (UPBs) that are locally irreducible in every bipartition in <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>d</mml:mi><mml:mo>&amp;#x2297;</mml:mo><mml:mi>d</mml:mi><mml:mo>&amp;#x2297;</mml:mo><mml:mi>d</mml:mi></mml:math> for any <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mi>d</mml:mi><mml:mo>&amp;#x2265;</mml:mo><mml:mn>3</mml:mn></mml:math>, and <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mn>3</mml:mn><mml:mo>&amp;#x2297;</mml:mo><mml:mn>3</mml:mn><mml:mo>&amp;#x2297;</mml:mo><mml:mn>3</mml:mn></mml:math> achieves the minimum dimension for the existence of such UPBs. These UPBs exhibit the phenomenon of strong quantum nonlocality without entanglement. Our result solves an open question given by Halder et al. [Phys. Rev. Lett. 122, 040403 (2019)] and Yuan et al. [Phys. Rev. A 102, 042228 (2020)]. It also sheds new light on the connections between UPBs and strong quantum nonlocality.