Reviewerly/exHarmony huggingface dataset:
Exploring Progressive Hybrid-Degraded Image Processing for Homography Estimation

Recent studies have shown that machine models do not coincide with the human perception of image quality, making mainstream image enhancement methods not always compatible with downstream tasks. To ameliorate this issue, this paper targets homography estimation, which is a fundamental step in image interpretation, to explore a hybrid-degraded image enhancement approach. Specifically, we build a reinforcement learning image processing framework with multiple lightweight tools, and develop a two-policy agent to progressively handle hybrid-degraded images. To match homography estimators with different properties, we further propose an environmental epistemic model (EEM) to build task-specific prior knowledge of uncertain environments. During the training process, the EEM is updated online and used to guide the agent’s exploration and exploitation. Comprehensive experiments are conducted on the aerial dataset where images are degraded from four visual perspectives: brightness, contrast, sharpness and stimulus. Results show that our agent can be trained to generate high-quality images for both learning-based and traditional homography estimators.
Contrastive Learning of Sentence Embeddings in Product Search

Contrastive learning has been widely applied for learning effective sentence embeddings for continue pretraining in domain-specific scenarios. Recent approaches in the product search domain have mainly focused on embedding-based retrieval, in which the state-of-the-art contrastive learning approach, unsupervised SimCSE, is usually employed. Ordinarily, as we know, the supervised SimCSE outperforms the unsupervised SimCSE, owing to the incorporation of annotated pairs as positives and hard negatives. In this paper, we propose WS-SimCSE, a weak supervision approach based on graph neural networks, which utilizes user behavior data to model relevance relationship between queries and items in a heterogeneous graph. With the neighborhood information of nodes with the same type, the positive pairs and hard negatives can be constructed for training objectives of supervised contrastive learning. Comparison evaluation experiments on several downstream benchmarks from a real-world online tourism platform demonstrate the robustness and effectiveness of WS-SimCSE.
CD-FSOD: A Benchmark For Cross-Domain Few-Shot Object Detection

In this paper, we propose a study of the cross-domain few-shot object detection (CD-FSOD) benchmark, consisting of image data from a diverse data domain. On the proposed benchmark, we evaluate state-of-art FSOD approaches, including meta-learning FSOD approaches and fine-tuning FSOD approaches. The results show that these methods tend to fall, and even underperform the naive fine-tuning model. We analyze the reasons for their failure and introduce a strong baseline that uses a mutually-beneficial manner to alleviate the overfitting problem. Our approach is remarkably superior to existing approaches by significant margins (2.0% on average) on the proposed benchmark. Our code is available at https://github.com/FSOD/CD-FSOD.
CRFAST: Clip-Based Reference-Guided Facial Image Semantic Transfer

This paper presents a new task for CLIP-based reference-guided facial image semantic transfer: the source facial image is translated to the output image with the high-level semantic attributes from the reference image while maintaining identity preservation. To this end, we employ the powerful generative capability of StyleGAN generator and the rich semantic knowledge of CLIP encoder to accomplish such a task. Additionally, a novel contrastive loss is designed to comprehensively explore the rich semantic information of CLIP for facial semantic concepts. This loss guides the semantic transfer toward desired directions from different perspectives in the pre-defined CLIP space. Besides, a simple yet effective semantic-preserved modulation module is proposed to explicitly map CLIP embeddings of reference image to the latent space. Experiments demonstrate that our approach achieves realistic facial image semantic transfer driven by reference images with various facial semantics.
Improving Adversarial Robustness with Hypersphere Embedding and Angular-Based Regularizations

Adversarial training (AT) methods have been found to be effective against adversarial attacks on deep neural networks. Many variants of AT have been proposed to improve its performance. Pang et al. [1] have recently shown that incorporating hypersphere embedding (HE) into the existing AT procedures enhances robustness. We observe that the existing AT procedures are not designed for the HE framework, and thus fail to adequately learn the angular discriminative information available in the HE framework. In this paper, we propose integrating HE into AT with regularization terms that exploit the rich angular information available in the HE framework. Specifically, our method, termed angular-AT, adds regularization terms to AT that explicitly enforce weight-feature compactness and inter-class separation; all expressed in terms of angular features. Experimental results show that angular-AT further improves adversarial robustness.
Improving Bert Fine-Tuning via Stabilizing Cross-Layer Mutual Information

Fine-tuning pre-trained language models, such as BERT, has shown enormous success among various NLP tasks. Though simple and effective, the process of fine-tuning has been found unstable, which often leads to unexpected poor performance. To increase stability and generalizability, most existing works resort to maintaining the parameters or representations of pre-trained models during fine-tuning. Nevertheless, very little work explores mining the reliable part of pre-learned information that can help to stabilize fine-tuning. To address this challenge, we introduce a novel solution in which we fine-tune BERT with stabilized cross-layer mutual information. Our method aims to preserve the reliable behaviors of cross-layer information propagation, instead of preserving the information itself, of the pre-trained model. Therefore, our method circumvents the domain conflicts between pre-trained and target tasks. We conduct extensive experiments with popular pre-trained BERT variants on NLP datasets, demonstrating the universal effectiveness and robustness of our method.
Effective Training of RNN Transducer Models on Diverse Sources of Speech and Text Data

This paper proposes a novel modeling framework for effective training of end-to-end automatic speech recognition (ASR) models on various sources of data from diverse domains: speech paired with clean ground truth transcripts, speech with noisy pseudo transcripts from semi-supervised decodes and unpaired text-only data. In our proposed approach, we build a recurrent neural network transducer (RNN-T) model with a shared multimodal encoder, multi-branch prediction networks and a shared common joint network. To train on unpaired text-only data sets along with transcribed speech data, the shared encoder is trained to process both speech and text modalities. Differences in data from multiple domains are effectively handled by training a multi-branch prediction network on various different data sets before an interpolation step combines the multi-branch prediction networks back into a computationally-efficient single branch. We show the benefit of our proposed technique on several ASR test sets by comparing our models to those trained by simple data mixing. The technique provides a significant relative improvement of up to 6% over baseline systems operating at a similar decoding cost.
Learnable Flow Model Conditioned on Graph Representation Memory for Anomaly Detection

Anomaly detection could be applied in a wide range of fields from industrial scene to medical imaging analysis. Although invertible flow models are developed to accomplish unsupervised anomaly detection, they are usually hard to train and have limited capabilities of accurately modeling the distribution of normal samples. To address this problem, we propose a novel enhanced flow model conditioned on graph representation memory (FlowGRM) for visual surface defect detection. In our FlowGRM, a graph neural network is trained to query graph embedding features from memory bank and incorporate them into a learnable flow model as conditional information. Such memorized conditional information, together with the invertible flow model, is jointly optimized to give rise to a probability density score. Experimental results obtained on the industrial visual anomaly detection datasets including MVTec and MTD show that the proposed model has state-of-the-art performance, which could even reach up to 99.6% and 99.2% AUROC scores on the above two bench-marks, respectively. Meanwhile, it requires fewer parameters and saves over 80% training time compared to existing flow models.
Anomaly Detection in Optical Spectra VIA Joint Optimization

Despite the remarkable progress of fiber optics in communication, little attention has been devoted to the automatic detection of anomalies in optical spectra, i.e., poorly transmitted channels. This task is typically addressed by ad-hoc heuristics that fall short in spectra presenting heavy distortions caused by optical amplifiers during transmission. We propose a method based on a joint optimization procedure for estimating the major trends that characterize the spectrum, enabling the detection of anomalies even in the presence of few channels and heavy distortions. Our experiments have shown that the proposed method can successfully localize anomalies achieving more than 98% accuracy, outperforming all competitors.
Retiformer: Retinex-Based Enhancement In Transformer For Low-Light Image

Transformer-based methods have shown impressive potential in many low-level vision tasks but are rarely used for low-light image enhancement (LLIE). Direct use of Transformer in LLIE will bring unnatural visual effects. This phenomenon encourages us to attempt to learn from the theory of Retinex. After trial and analysis, we finally propose Retiformer. Retiformer decomposes images into reflectance and illumination attention maps by Retinex Window Self-Attention (R-WSA). It will replace element-wise multiplication with the attention mechanism. By the R-WSA, we respectively apply a Decom-Retiformer block and an Enhance-Retiformer block at the head and tail of a Transformer-based backbone. They can decompose and align the reflection and illumination components just like RetinexNet. With this pipeline, Retiformer combines the advantages of Transformer and Retinex theory and achieves state-of-the-art performance of Retinex-based methods.
Semantic Preprocessor for Image Compression for Machines

Visual content is being increasingly transmitted and consumed by machines rather than humans to perform automated content analysis tasks. In this paper, we propose an image preprocessor that optimizes the input image for machine consumption prior to encoding by an off-the-shelf codec designed for human consumption. To achieve a better trade-off between the accuracy of the machine analysis task and bitrate, we propose leveraging pre-extracted semantic information to improve the preprocessor’s ability to accurately identify and filter out task-irrelevant information. Furthermore, we propose a two-part loss function to optimize the preprocessor, consisted of a rate-task performance loss and a semantic distillation loss, which helps the reconstructed image obtain more information that contributes to the accuracy of the task. Experiments show that the proposed preprocessor can save up to 48.83% bitrate compared with the method without the preprocessor, and save up to 36.24% bitrate compared to existing preprocessors for machine vision.
Self-Attention Based Action Segmentation Using Intra-And Inter-Segment Representations

Segmenting activities in untrimmed videos remains a critical challenge to fully understand complex human activity sequences. A correct representation of temporal action relations is key for improving incorrect segmentations. We propose a self-attention-based model that refines initial segmentations by separately considering intra-as well as inter-segment relations between predicted action segments. Furthermore, in order to enhance the training process, we use a similarity-guided regularization technique that ensures intra-segment similarity and the validity of action transitions between adjacent segments. In an extensive evaluation on three public datasets -Georgia Tech Egocentric Activities, 50Salads, and Breakfast -our proposed architecture enhances the backbone model by 6.1% on GTEA, 3.8% on 50Salads, and 3.9% on Breakfast with regard to the F 1@50 metric.
MFAT: A Multi-Level Feature Aggregated Transformer for Person Re-Identification

Recently, with the development of the Transformer, re-identification (ReID) has great success in various applications. Existing works prefer to utilize the Transformer’s highest-level information as its discriminative feature, which focuses on a few concentrated parts or areas. However, in ReID filed, under such various scenes and camera views, only using a few concentrated parts to distinguish the query person is insufficient. Meanwhile, we find that Transformer’s lower-level information is also helpful for the recognition accuracy of the query person, especially, when the scene changes greatly. Therefore, we propose a Multi-level Feature Aggregated Transformer for person re-identification (MFAT) with high performance. To aggregate multi-level information, two novel modules are carefully designed. (i) The Global Content and Structure Aggregation (GCSA) module is proposed to aggregate multi-level information in a global manner. (ii) The Local Convolution Aggregation (LCA) module which consists of a series of convolutional blocks, is introduced to aggregate multi-level features with local operations. To the best of our knowledge, this is the first work to aggregate multi-level features with a Transformer backbone for person ReID task. Experiment results show that our method has achieved state-of-the-art on three person ReID benchmarks, with both Pyramid Vision Transformer (PVT) and Vision Transformer (ViT) backbones.
HEiMDaL: Highly Efficient Method for Detection and Localization of Wake-Words

Streaming keyword spotting is a widely used solution for activating voice assistants. Methods based on Deep Neural Networks with Hidden Markov Model (DNN-HMM) have proven to be efficient and widely adopted in this space, primarily because of the ability to detect and identify the start and end of the wake-up word at low compute cost. However, such hybrid systems suffer from loss metric mismatch when the DNN and HMM are trained independently. Sequence discriminative training cannot fully mitigate the loss-metric mismatch due to the inherent Markovian style of the operation. We propose an low footprint CNN model, called HEiMDaL, to detect and localize keywords in streaming conditions. We introduce an alignment-based classification loss to detect the occurrence of the keyword along with an offset loss to predict the start of the keyword. HEiMDaL shows 73% improvement in detection metrics along with equivalent localization accuracy and with the same memory footprint as existing DNN-HMM style models for a given wake-word.
Multilevel Authentication in Smart Online Classroom Attendance

This project presents a prototype of a multilevel authentication in smart online classroom attendance. The scope of this project includes some steps. It begins with conceptual design of a multilevel authentication for students’ attendance in the online classroom application. The system generates a student unique identity for the first-level authentication followed by a biometric authentication for student verification. Overall, the development of a multilevel authentication scheme for the online classroom is presented in this paper. The prototype can authenticate the student unique identity and biometric authentication for student verification. In addition, the designed smart system prototype is also able to store the student attendance and sending back to the lecturer.
Pengaruh intensitas membaca dan tingkat keseringan membaca karya sastra terhadap keterampilan menulis kreatif siswa

Penelitian ini bertujuan mengungkapkan: (1) pengaruh intensitas membaca; (2) pengaruh tingkat keseringan membaca karya sastra; (3) pengaruh intensitas membaca dan tingkat keseringan membaca karya sastra; (4) mana yang lebih berpengaruh di antara intensitas membaca dan tingkat keseringan membaca karya sastra terhadap keterampilan menulis kreatif siswa SMA. Jenis penelitian ini adalah ex-post facto. Populasi penelitian ini seluruh siswa SMA negeri di Kota Yogyakarta. Teknik pengambilan sampel yang digunakan proportionate stratified random sampling. Instrumen penelitian yang digunakan angket dan tes. Data dianalisis dengan teknik statistik korelasi parsial yang diteruskan dengan regresi ganda. Hasil penelitian menunjukkan bahwa (1) intensitas membaca secara signifikan berpengaruh terhadap keterampilan menulis kreatif siswa dengan nilai r 0,977 p 0,05; (2) tingkat keseringan membaca karya sastra berpengaruh secara signifikan terhadap keterampilan menulis kreatif siswa dengan nilai r 0,990 p 0,05; (3) intensitas membaca dan tingkat keseringan membaca karya sastra berpengaruh terhadap keterampilan menulis kreatif siswa dengan Fhitung 6623,724 p 0,05; (4) variabel tingkat keseringan membaca karya sastra lebih berpengaruh secara signifikan daripada variabel intensitas membaca terhadap keterampilan menulis kreatif siswa dibuktikan dengan nilai standardized coeficients beta 0,161 1,149. The effect of reading intensity and frequency of reading literature on creative writing skill AbstractThis research aims to reveal: (1) the effect of reading intensity; (2) the effect of frequency of reading literature; (3) the effect of reading intensity and the frequency of reading literature; (4) which is more influential between reading intensity and the frequency of reading literature to the creative writing skills of state senior high school students. This research was an ex-post facto. The proportional stratified random sampling technique was utilized to establish the sample. The research instruments were questionnaires and tests. The data were analyzed using the partial regression technique continued with multiple regression. The results of the study showed that: (1) reading intensity significantly affects creative writing skill of state senior high school student with r value = 0,977 p 0,05; (2) frequency of reading literature significantly affects creative writing skill of state senior high school students with r value = 0.990 p 0.05; (3) reading intensity and frequency of reading literature affect creative writing skills of state senior high school students with Fobtained = 6623,724 p 0,05; (4) the variable of frequency of reading literature is significantly more affects students' creative writing skill than reading intensity variable. This is indicated by the value of standardized coeficients beta 0.161 1.149.
Developing multimedia for reading instruction

This research study aimed to (1) develop multimedia for reading instruction for the ninth-grade students at SMP N 1 Kalasan, and (2) find out the quality of multimedia for reading instruction for the ninth-grade students at SMP N 1 Kalasan. This research study applied research and development adapted from Lee Owens (2004). The development of multimedia in this study was conducted in four stages: analysis, design, development, and evaluation and revision. Data were obtained through questionnaires. The collected data were analyzed by means of descriptive statistics. The findings showed that the quality of the developed multimedia for reading instruction for the ninth-grade students at SMP N 1 Kalasan was “good” in terms of instructional and material aspects (with average scores 4.17 and 4.10) respectively, and “excellent” in terms of media aspect (average score 4.23). In terms of those three aspects, the developed multimedia was categorized as “good” with an average score of 4.16.
SODA: A Set of Fast Oblivious Algorithms in Distributed Secure Data Analytics

Cloud systems are now a prevalent platform to host large-scale big-data analytics applications such as machine learning and relational database. However, data privacy remains as a critical concern for public cloud systems. Existing trusted hardware could provide an isolated execution domain on an untrusted platform, but also suffers from access-pattern-based side channels at various levels including memory, disks, and networking. Oblivious algorithms can address these vulnerabilities by hiding the program data access patterns. Unfortunately, current oblivious algorithms for data analytics are limited to single-machine execution, only support simple operations, and/or suffer from significant performance overheads due to the use of expensive global sort and excessive data padding. In this work, we propose SODA, a set of efficient and oblivious algorithms for distributed data analytics operators, including filter, aggregate, and binary equi-join. To improve performance, SODA completely avoids the expensive oblivious global sort primitive, and minimizes the data padding overheads. SODA makes use of low-cost (pseudo-)random communication instead of expensive global sort to ensure uniform data traffic in oblivious filter and aggregate. It also adopts a novel two-level bin-packing approach in oblivious join to alleviate both input redistribution and join product skewness, thus minimizing necessary data padding. Compared to the state-of-the-art system, SODA not only extends the functionality but also improves the performance. It achieves 1.1× to 14.6× speedups on complex multi-operator data analytics workloads.
Neighboring-Part Dependency Mining and Feature Fusion Network for Person Re-Identification

Person re-identification (Re-ID) is a computer vision technique used to determine the presence of a specific pedestrian target in an image or video sequence. It is an important branch of image retrieval. With the advancements in deep learning, notable progress has been achieved in Re-ID research. However, existing methods primarily focus on the most prominent features in the image, ignoring other less obvious yet beneficial features and spatial interdependencies within the image. To address this issue, this paper proposes a neighboring-part dependency mining and feature fusion network (NDMF-Net). The network horizontally splits pedestrian features into multiple parts, using a part-level hybrid attention module (PHAM) to focus on the salient region of each part, and a neighboring-part dependency exploration module (NDEM) to extract the spatial dependencies between neighboring parts of the image. Eventually, different features are fused to form the final representation. We validate the NDMF-Net on mainstream datasets and the experimental results demonstrate that our method is effective and achieves state-of-the-art performance.
Improved Differential Privacy Noise Mechanism in Quantum Machine Learning

Quantum computing, as an emerging research field, is attracting people’s attention. It has been proven to be superior in many ways to classical computing. Differential privacy provides an easy way to achieve demonstrable privacy, and the most common method is to add noise to datasets. At this stage of quantum computers, noise is a factor that cannot be ignored. Indeed, the existence of noise will negatively affect the performance of quantum computers, but we can apply it to privacy protection. In this work, we will consider two situations: inherent noise and artificially added noise. The noise is added to the variational quantum algorithm to implement quantum machine learning with differential privacy. The importance of each type will be examined and less artificial noise is needed for a common privacy budget over classical machine learning. Composition theory will be invoked to prove the advantage of the entire quantum machine learning process.