Reviewerly/exHarmony huggingface dataset:
Towards Making a Trojan-Horse Attack on Text-to-Image Retrieval

While deep learning based image retrieval is reported to be vulnerable to adversarial attacks, existing works are mainly on image-to-image retrieval with their attacks performed at the front end via query modification. By contrast, we present in this paper the first study about a threat that occurs at the back end of a text-to-image retrieval (T2IR) system. Our study is motivated by the fact that the image collection indexed by the system will be regularly updated due to the arrival of new images from various sources such as web crawlers and advertisers. With malicious images indexed, it is possible for an attacker to indirectly interfere with the retrieval process, letting users see certain images that are completely irrelevant w.r.t. their queries. We put this thought into practice by proposing a novel Trojan-horse attack (THA). In particular, we construct a set of Trojan-horse images by first embedding word-specific adversarial information into a QR code and then putting the code on benign advertising images. A proof-of-concept evaluation, conducted on two popular T2IR datasets (Flickr30k and MS-COCO), shows the effectiveness of the proposed THA in a white-box mode.
Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognition

Adapting a trained Automatic Speech Recognition (ASR) model to new tasks results in catastrophic forgetting of old tasks, limiting the model’s ability to learn continually and to be extended to new speakers, dialects, languages, etc. Focusing on End-to-End ASR, in this paper, we propose a simple yet effective method to overcome catastrophic forgetting: weight averaging. By simply taking the average of the previous and the adapted model, our method achieves high performance on both the old and new tasks. It can be further improved by introducing a knowledge distillation loss during the adaptation. We illustrate the effectiveness of our method on both monolingual and multilingual ASR. In both cases, our method strongly outperforms all baselines, even in its simplest form.
Phoneme-Level Bert for Enhanced Prosody of Text-To-Speech with Grapheme Predictions

Large-scale pre-trained language models have been shown to be helpful in improving the naturalness of text-to-speech (TTS) models by enabling them to produce more naturalistic prosodic patterns. However, these models are usually word-level or sup-phoneme-level and jointly trained with phonemes, making them inefficient for the downstream TTS task where only phonemes are needed. In this work, we propose a phoneme-level BERT (PL-BERT) with a pretext task of predicting the corresponding graphemes along with the regular masked phoneme predictions. Subjective evaluations show that our phoneme-level BERT encoder has significantly improved the mean opinion scores (MOS) of rated naturalness of synthesized speech compared with the state-of-the-art (SOTA) StyleTTS baseline on out-of-distribution (OOD) texts.
Quantpipe: Applying Adaptive Post-Training Quantization For Distributed Transformer Pipelines In Dynamic Edge Environments

Pipeline parallelism has achieved great success in deploying large-scale transformer models in cloud environments, but has received less attention in edge environments. Unlike in cloud scenarios with high-speed and stable network inter-connects, dynamic bandwidth in edge systems can degrade distributed pipeline performance. We address this issue with QuantPipe, a communication-efficient distributed edge system that introduces post-training quantization (PTQ) to compress the communicated tensors. QuantPipe uses adaptive PTQ to change bitwidths in response to bandwidth dynamics, maintaining transformer pipeline performance while incurring limited inference accuracy loss. We further improve the accuracy with a directed-search analytical clipping for integer quantization method (DS-ACIQ), which bridges the gap between estimated and real data distributions. Experimental results show that QuantPipe adapts to dynamic bandwidth to maintain pipeline performance while achieving a practical model accuracy using a wide range of quantization bitwidths, e.g., improving accuracy under 2-bit quantization by 15.85% on ImageNet compared to naive quantization.
Semi-Swinderain: Semi-Supervised Image Deraining Network Using SWIN Transformer

Currently, single image deraining lacks paired rain/clean images in real world and most studies use synthetic data. Real rain image deraining is still a challenge. To solve this problem, we propose a semi-supervised image deraining network using Swin Transformer, which can both use features of synthetic data and real data to get a better result. Specifically, the network is divided into supervised branch and unsupervised branch. Supervised and unsupervised branches are trained using synthetic data and real data, respectively. The network architecture is based on Swin Transformer, which adds a self-supervised memory module between encoder and decoder to store rain information. In the unsupervised branch, contrastive loss is added to ensure restored real rain image in features space is close to clear image, away from real rain image. In addition, we propose a real rain dataset RealRain11k. Experiments show our method has better result in real rain image deraining. The source code and RealRain11k are available at https://github.com/imissrc/Semi-SwinDerain.
MariusGNN: Resource-Efficient Out-of-Core Training of Graph Neural Networks

We study training of Graph Neural Networks (GNNs) for large-scale graphs. We revisit the premise of using distributed training for billion-scale graphs and show that for graphs that fit in main memory or the SSD of a single machine, out-of-core pipelined training with a single GPU can outperform state-of-the-art (SoTA) multi-GPU solutions. We introduce MariusGNN, the first system that utilizes the entire storage hierarchy---including disk---for GNN training. MariusGNN introduces a series of data organization and algorithmic contributions that 1) minimize the end-to-end time required for training and 2) ensure that models learned with disk-based training exhibit accuracy similar to those fully trained in memory. We evaluate MariusGNN against SoTA systems for learning GNN models and find that single-GPU training in MariusGNN achieves the same level of accuracy up to 8× faster than multi-GPU training in these systems, thus, introducing an order of magnitude monetary cost reduction. MariusGNN is open-sourced at www.marius-project.org.
Open-Set Automatic Target Recognition

Automatic Target Recognition (ATR) is a category of computer vision algorithms which attempts to recognize targets on data obtained from different sensors. ATR algorithms are extensively used in real-world scenarios such as military and surveillance applications. Existing ATR algorithms are developed for traditional closed-set methods where training and testing have the same class distribution. Thus, these algorithms have not been robust to unknown classes not seen during the training phase, limiting their utility in real-world applications. To this end, we propose an Open-set Automatic Target Recognition framework where we enable open-set recognition capability for ATR algorithms. In addition, we introduce a plugin Category-aware Binary Classifier (CBC) module to effectively tackle unknown classes seen during inference. The proposed CBC module can be easily integrated with any existing ATR algorithms and can be trained in an end-to-end manner. Experimental results show that the proposed approach outperforms many open-set methods on the DSIAC and CIFAR-10 datasets. To the best of our knowledge, this is the first work to address the open-set classification problem for ATR algorithms. Source code is available at: https://github.com/bardisafa/Open-set-ATR.
Progressive Meta-Pooling Learning for Lightweight Image Classification Model

Practical networks for edge devices adopt shallow depth and small convolutional kernels to save memory and computational cost, which leads to a restricted receptive field. Conventional efficient learning methods focus on lightweight convolution designs, ignoring the role of the receptive field in neural network design. In this paper, we propose the Meta-Pooling framework to make the receptive field learnable for a lightweight network, which consists of parameterized pooling-based operations. Specifically, we introduce a parameterized spatial enhancer, which is composed of pooling operations to provide versatile receptive fields for each layer of a lightweight model. Then, we present a Progressive Meta-Pooling Learning (PMPL) strategy for the parameterized spatial enhancer to acquire a suitable receptive field size. The results on the ImageNet dataset demonstrate that MobileNetV2 using Meta-Pooling achieves top1 accuracy of 74.6%, which outperforms MobileNetV2 by 2.3%.
Align, Write, Re-Order: Explainable End-to-End Speech Translation via Operation Sequence Generation

The black-box nature of end-to-end speech-to-text translation (E2E ST) makes it difficult to understand how source language inputs are being mapped to the target language. To solve this problem, we propose to simultaneously generate automatic speech recognition (ASR) and ST predictions such that each source language word is explicitly mapped to a target language word. A major challenge arises from the fact that translation is a non-monotonic sequence transduction task due to word ordering differences between languages – this clashes with the monotonic nature of ASR. Therefore, we propose to generate ST tokens out-of-order while remembering how to re-order them later. We achieve this by predicting a sequence of tuples consisting of a source word, the corresponding target words, and post-editing operations dictating the correct insertion points for the target word. We examine two variants of such operation sequences which enable generation of monotonic transcriptions and non-monotonic translations from the same speech input simultaneously. We apply our approach to offline and real-time streaming models, demonstrating that we can provide explainable translations without sacrificing quality or latency. In fact, the delayed re-ordering ability of our approach improves performance during streaming. As an added benefit, our method performs ASR and ST simultaneously, making it faster than using two separate systems to perform these tasks.
Sparse Graph Learning with Spectrum Prior for Deep Graph Convolutional Networks

A graph convolutional network (GCN) employs a graph filtering kernel tailored for data with irregular structures. However, simply stacking more GCN layers does not improve performance; instead, the output converges to an uninformative low-dimensional subspace, where the convergence rate is characterized by the graph spectrum— this is the known over-smoothing problem in GCN. In this paper, we propose a sparse graph learning algorithm incorporating a new spectrum prior to compute a graph topology that circumvents over-smoothing while preserving pairwise correlations inherent in data. Specifically, based on a spectral analysis of multilayer GCN output, we derive a spectrum prior for the graph Laplacian matrix L to robustify the model expressiveness against over-smoothing. Then, we formulate a sparse graph learning problem with the spectrum prior, solved efficiently via block coordinate descent (BCD). Moreover, we optimize the weight parameter trading off the fidelity term with the spectrum prior, based on data smoothness on the original graph learned without spectrum manipulation. The output L is then normalized for supervised GCN training. Experiments show that our proposal produced deeper GCNs and higher prediction accuracy for regression and classification tasks compared to competing schemes.
Two-Stream Decoder Feature Normality Estimating Network for Industrial Anomaly Detection

Image reconstruction-based anomaly detection has recently been in the spotlight because of the difficulty of constructing anomaly datasets. These approaches work by learning to model normal features without seeing abnormal samples during training and then discriminating anomalies at test time based on the reconstructive errors. However, these models have limitations in reconstructing the abnormal samples due to their indiscriminate conveyance of features. Moreover, these approaches are not explicitly optimized for distinguishable anomalies. To address these problems, we propose a two-stream decoder network (TSDN), designed to learn both normal and abnormal features. Additionally, we propose a feature normality estimator (FNE) to eliminate abnormal features and prevent high-quality reconstruction of abnormal regions. Evaluation on a standard benchmark demonstrated performance better than state-of-the-art models.
MCKD: Mutually Collaborative Knowledge Distillation For Federated Domain Adaptation And Generalization

Conventional unsupervised domain adaptation (UDA) and domain generalization (DG) methods rely on the assumption that all source domains can be directly accessed and combined for model training. However, this centralized training strategy may violate privacy policies in many real-world applications. A paradigm for tackling this problem is to train multiple local models and aggregate a generalized central model without data sharing. Recent methods have made remarkable advancements in this paradigm by exploiting parameter alignment and aggregation. But when sources domain variety increases, directly aligning and aggregating local parameters becomes more challenging. Adapting a different approach in this work, we devised a data-free semantic collaborative distillation strategy to learn domain-invariant representation for both federated UDA and DG. Each local model transmits its predictions to the central server and derives its target distribution from the average of other local models' distributions to facilitate the mutual transfer of domain-specific knowledge. When unlabeled target data is available, we introduce a novel UDA strategy termed knowledge filter to adapt the central model to the target data. Extensive experiments on four UDA and DG datasets demonstrate that our method has a competitive performance compared with the state-of-the-art methods.
Automatic Attendance Monitoring System Using LBPH and HAAR Algorithm

Every office and organization must have an attendance monitoring system. Every institution has a standard attendance policy that requires the professor or assistant to report the student's name and attendance for each and every session. An automatic attendance monitoring approach will be suggested to replace this human process. Face spotting, an image processing tool, and face identification techniques are used in the research project that is being suggested. This method assists in turning video frames into numbers so that the human profile may be quickly recognized for attendance monitoring. The image captured from the video should be compared with the data saved with the date and time, which will be automatically stored in the memory storage, before a recognized attendance database is formed. The Viola-Jones Face Detection Technique, also known as the HAAR cascade algorithm, and the Local Binary Pattern Histogram (LBPH) algorithm were used in the development of the proposed automatic face recognition attendance monitoring system. Their combined features are used for attendance monitoring. The human face is first captured using the HAAR cascade technique, and then the LBPH algorithm is utilized to save the distinctive traits of each human face. During the face identification procedure, a predetermined user ID is retrieved from the algorithm data. This will produce the student's attendance data for their presence. The image processing system's data will need to be kept up to date and stored daily. The implementation of an automatic attendance tracking system will benefit from the suggested strategy. Anaconda software is used to implement and analyse the suggested attendance tracking system.
A Critical Analysis of the Block Chain in Manufacturing System Implementation

As component of the industry 4.0 project, a novel concept of Cyber-Physical Systems (CPS), fog computing, big data analytics, cloud manufacturing, the Internet of Things (IoT), and other technologies have been brought to the manufacturing sector. Some of the projected benefits and possibilities that these innovations might provide include self-prediction, self-maintenance, self-comparison, self-configuration, and self-awareness. In these concepts, the centralised corporate system and third-party trust operations are still in use. However, contemporary manufacturing has a slew of problems, including data security and reliability, adaptability, security, trust, and privacy. This paper provides an analysis of the Blockchain Technology (BT), reviews the application of blockchain in Artificial Intelligence (AI), proposes a model for blockchain-enabled CPS and discusses the potential challenges that face the implementation of blockchain in manufacturing systems.
A Build and Deploy Ethereum Smart Contract for Food Supply Chain Management in Truffle - Ganache Framework

Blockchain innovation is a disseminated information base of records. It’s perhaps, troublesome innovations for the layout, tasks, and fundamental operation of supply chains. But, the blockchain helps a lot in supply chain management to bring the trustworthiness (tracking and tracing) to many parameters. Additionally, Food Supply Chain Management. It (Blockchain) helps a lot to avoid food contamination while transactions. Ethereum Brilliant deals, as built rules between the parties to maintain the protocols without any violation. Because it’s built-in computerized (code) so no security breaches between parties, This, Brilliant agreements open system reports dividing among inventory network members, and consistent cycle improvement. Truffle Suite, Ganache guarantees a detached individual blockchain network you can use for testing brilliant agreements. The functionalities of Ganache empower improved efficiency concerning the velocity of testing and sending contracts.
AI Agents at Different Data Centers to Minimize the Energy Spending

The proposed system develops and evaluates machine learning models for predicting power consumption in data centers. The project involves data collection and preprocessing, training and evaluation of five models including Long Short-Term Memory (LSTM), Deep Neural Network (DNN), Extreme Gradient Boosting (XGB), Gradient Boosting (GB), and K-Nearest Neighbors (KNN), and the comparison of their performances using Mean Squared Error (MSE). The best performing model is deployed in a production environment and continuously monitored for improvement. The proposed system utilizes AI and deep learning technology to make accurate energy consumption predictions based on real-time data, eliminating the need for manual monitoring and improving accuracy and efficiency. The system also provides an efficient approach to managing energy consumption while maintaining security, reducing the risks associated with manual management.
Channel Estimation and Signal Detection in OFDM Systems using Deep Learning

This paper presents results on deep learning-based signal recognition and channel estimation using orthogonal frequency-division multiplexing (OFDM) systems. Here, deep learning is used to fully regulate wireless OFDM channels. Instead of estimating CSI explicitly before identifying or recovering the broadcast symbols using the estimated CSI, as is the case with standard OFDM receivers, The suggested deep learning-based solution directly recovers the transmitted symbols while implicitly estimating channel state information (CSI). A deep learning model is initially trained offline using data creation in order to eliminate channel distortion based on channel characteristics, and it is then utilized to directly extract the live transmitted data. The simulation outcomes exhibit the potential of the deep learning-based method can identify transmitted symbols and correct for channel distortion with a level of performance equivalent to the minimum mean square error (MMSE) estimator. Less training pilots, the removal of the cyclic prefix (CP), and the presence of nonlinear clipping noise make deep learning approaches more trustworthy than conventional procedures. For channel estimation and signal recognition deep learning is an effective method in wireless communications characterized by intricate channel distortion and interference.
A Concept-based Ontology Mapping Method for Effective Retrieval of Bio-Medical Documents

Nowadays, document retrieval is one of the major tasks in Biomedical Research. Due to the size, a lot of time gets wasted for the user in searching the information they need while retrieving the documents, the problem that arises is the retrieval of non-related documents. In order to overcome this, a concept based ontology mapping method is proposed in this paper. Initially, a query document is given to the retrieval system for retrieving the matched documents based on the concepts in the document. A Stanford parser is used for making the document into sentences and then into Parts-of-speech tagger and Context-free phrase structure grammar representation. After that, the resultant noun phrases and verb phrases are considered as concepts and linking phrases and from these the concept maps are generated for each document. Then the concepts from the concept map of query document are matched with the concept maps of the target document collection using Word Net Ontology. If the measure for the matching is less than the threshold value means, the alternate documents are checked for the matching. Otherwise, the relevant documents for the matched query is retrieved based on the concept maps from the document storage. The experimentation was carried out with the help of medical documents. The proposed retrieval system shows that the documents are retrieved effectively based on concepts and the system performs with higher accuracy value.