Reviewerly/exHarmony huggingface dataset:
BL-JUNIPER: A CNN-Assisted Framework for Perceptual Video Coding Leveraging Block-Level JND

Just Noticeable Distortion (JND) finds the minimum distortion level perceivable by humans. This can be a natural solution for setting the compression for each video region in perceptual video coding. However, existing JND-based solutions estimate JND levels for each video frame and ignore the fact that different video regions have different perceptual importance. To address this issue, we propose a Block-Level Just Noticeable Distortion-based Perceptual (BL-JUNIPER) framework for video coding. The proposed four-stage framework combines different perceptual information to further improve the prediction accuracy. The JND mapping in the first stage derives block-level JNDs from frame-level information without the need to collect a new bock-level JND dataset. In the second stage, an efficient CNN-based model is proposed to predict JND levels for each block according to spatial and temporal characteristics. Unlike existing methods, BL-JUNIPER works on raw video frames and avoids re-encoding each frame several times, making it computationally practical. Third, the visual importance of each block is measured using a visual attention model. Finally, a proposed quantization control algorithm uses both JND levels and visual importance to adjust the Quantization Parameter (QP) for each block. The specific algorithm for each stage of the proposed framework can be changed, as long as the input and output formats of each block are followed, without the need to change other stages, based on any current or future methods, providing a flexible and robust solution. Extensive experimental results demonstrate that BL-JUNIPER achieves a mean bitrate reduction of 27.75% with a Delta Mean Opinion Score (DMOS) close to zero and BD-Rate gains of 25.44% based on MOS, compared to the baseline encoding, and also gains a better performance compared to competing methods.
Area-Time-Efficient Scalable Schoolbook Polynomial Multiplier for Lattice-Based Cryptography

Lattice-based cryptography (LBC) stands out as one of the most viable classes of quantum-resistant schemes. This brief explores a time-sharing approach, with different parallelism levels, for a crucial operation in LBC cryptosystems, i.e., polynomial multiplication. We also employ an innovative coefficient ordering method in our time-shared schoolbook polynomial multiplication (SPM) to combine the best of two worlds: design compactness and lower processing latency. Thus, our work offers a choice of design points with performance vs. resource trade-offs. Our fastest proposed design exhibits 80% and 57% reductions in LUTs and throughput, respectively, compared to the existing fully parallel SPM architecture (on Xilinx Ultrascale+), which lead to a 53% improvement in the area-time-product efficiency. Our smallest proposed design is more than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2.2\times $ </tex-math></inline-formula> faster than the existing low-cost parallel SPM architecture (on Xilinx Kintex-7) at the expense of 85% additional area resources.
Noise Robust Illumination Invariant Face Recognition Via Bivariate Wavelet Shrinkage in Logarithm Domain

Abstract Recognizing faces under various lighting conditions is a challenging problem in artificial intelligence and applications. In this paper we describe a new face recognition algorithm which is invariant to illumination. We first convert image files to the logarithm domain and then we implement them using the dual-tree complex wavelet transform (DTCWT) which yields images approximately invariant to changes in illumination change. We classify the images by the collaborative representation-based classifier (CRC). We also perform the following sub-band transformations: (i) we set the approximation sub-band to zero if the noise standard deviation is greater than 5; (ii) we then threshold the two highest frequency wavelet sub-bands using bivariate wavelet shrinkage. (iii) otherwise, we set these two highest frequency wavelet sub-bands to zero. On obtained images we perform the inverse DTCWT which results in illumination invariant face images. The proposed method is strongly robust to Gaussian white noise. Experimental results show that our proposed algorithm outperforms several existing methods on the Extended Yale Face Database B and the CMU-PIE face database.
Enhancing Unsupervised Video Representation Learning by Temporal Contrastive Modelling Using 2D CNN

Using video sequence order as a supervised signal has proven to be effective in initializing 2d convnets for downstream tasks like video retrieval and action recognition. Earlier works used it as sequence sorting task, odd-one out task and sequence order prediction task. In this work, we propose an enhanced unsupervised video representation learning method by solving order prediction and contrastive learning jointly using 2d-CNN (as backbone). With contrastive learning we aim to pull different temporally transformed versions of same video sequence closer while pushing the other sequences away in the latent space. In addition, instead of pair wise feature extraction, the features are learned with 1-d temporal convolutions. Experiments conducted on UCF-101 and HMDB-51 datasets show that our proposal outperforms the other methods on both down-stream tasks (video retrieval and action recognition) with 2d-CNN and, achieves satisfactory results compared to 3d-CNN based methods.
Centroid based person detection using pixelwise prediction of the position

Implementations of person detection in tracking and counting systems tend towards processing of orthogonally captured images on edge computing devices. The ellipse-like shape of heads in orthogonally captured images inspired us to predict head centroids to determine positions of persons in images. We predict the centroids using a fully convolutional network (FCN). We combine the FCN with simple image processing operations to ensure fast inference of the detector. We experiment with the size of the FCN output to further decrease the inference time. We compare the proposed centroid-based detector with bounding box-based detectors on head detection task in terms of the inference time and the detection performance. We propose a performance measure which allows quantitative comparison of the two detection approaches. For the training and evaluation of the detectors, we form original datasets of 8000 annotated images, which are characterized by high variability in terms of lighting conditions, background, image quality, and elevation profile of scenes. We propose an approach which allows simultaneous annotation of the images for both bounding box-based and centroid-based detection. The centroid-based detector shows the best detection performance while keeping edge computing standards.
Constructions of MDS symbol-pair codes with minimum distance seven or eight

Symbol-pair codes are proposed to guard against pair-errors in symbol-pair read channels. The minimum symbol-pair distance plays a vital role in determining the error-correcting capability and the constructions of symbol-pair codes with largest possible minimum symbol-pair distance is of great importance. Maximum distance separable (MDS) symbol-pair codes are optimal in the sense that such codes can acheive the Singleton bound. In this paper, for length 5p, two new classes of MDS symbol-pair codes with minimum symbol-pair distance seven or eight are constructed by utilizing repeated-root cyclic codes over $${\mathbb {F}}_{p}$$ , where p is a prime. In addition, we derive a class of MDS symbol-pair codes with minimum symbol-pair distance seven and length 4p.
Audio–visual collaborative representation learning for Dynamic Saliency Prediction

The Dynamic Saliency Prediction (DSP) task simulates the human selective attention mechanism to perceive a dynamic scene, which is significant and imperative in many vision tasks. Most of existing methods only consider visual cues, while neglecting the accompanying audio information, which can provide complementary information for the scene understanding. Indeed, there exists a strong relation between auditory and visual cues, and humans generally perceive the surrounding scene by collaboratively sensing these cues. Motivated by this, an audio–visual collaborative representation learning method is proposed for the DSP task, which explores the implicit knowledge in the audio modality to better predict the dynamic saliency map by assisting the visual modality. The proposed method consists of three parts: (1) audio–visual encoding, (2) audio–visual localization, and (3) collaborative integration parts. First, a refined SoundNet architecture is adopted to encode audio modality for obtaining corresponding features, and a modified 3D ResNet-50 architecture is employed to learn visual features, containing both spatial location and temporal motion information. Secondly, an audio–visual localization part is devised to locate the sounding salient object in the visual scene by learning the correspondence between audio and visual information. Thirdly, a collaborative integration part is devised to adaptively aggregate the audio–visual information and a center-bias prior to generate the final saliency map. Extensive experiments are conducted on six challenging audiovisual eye-tracking datasets, namely DIEM, AVAD, Coutrot1, Coutrot2, SumMe, and ETMD, which shows significant superiority over state-of-the-art DSP models.
Connecting Low-Loss Subspace for Personalized Federated Learning

Due to the curse of statistical heterogeneity across clients, adopting a personalized federated learning method has become an essential choice for the successful deployment of federated learning-based services. Among diverse branches of personalization techniques, a model mixture-based personalization method is preferred as each client has their own personalized model as a result of federated learning. It usually requires a local model and a federated model, but this approach is either limited to partial parameter exchange or requires additional local updates, each of which is helpless to novel clients and burdensome to the client's computational capacity. As the existence of a connected subspace containing diverse low-loss solutions between two or more independent deep networks has been discovered, we combined this interesting property with the model mixture-based personalized federated learning method for improved performance of personalization. We proposed SuPerFed, a personalized federated learning method that induces an explicit connection between the optima of the local and the federated model in weight space for boosting each other. Through extensive experiments on several benchmark datasets, we demonstrated that our method achieves consistent gains in both personalization performance and robustness to problematic scenarios possible in realistic services.
QCNet: query context network for salient object detection of automatic surface inspection

Abstract Building upon fully convolutional networks (FCNs), deep learning-based salient object detection (SOD) methods achieve gratifying performance in many vision tasks, including surface defect detection. However, most existing FCN-based methods still suffer from the coarse object edge predictions. The state-of-the-art methods employ intricate feature aggregation techniques to refine boundaries, but they are often too computational cost to deploy in the real application. This paper proposes a semantics guided detection paradigm for salient object detection. Guided atrous pyramid module is first applied on the top feature to segment complete salient semantics. Query context modules are further used to build relation maps between saliency and structural information from the top-down pathway. These two modules allow the semantic features to flow throughout the decoder phase, yielding detail enriched saliency predictions. Experimental results demonstrate that the proposed method performs favorably against the state-of-the-art methods on surface defect detection and SOD benchmarks. In addition, this method can detect at 27 FPS in a fully convolutional fashion without any post-processing, which has the potential for real-time detection.
A Novel Ensemble of Arithmetic Optimization Algorithm and Harris Hawks Optimization for Solving Industrial Engineering Optimization Problems

Recently, numerous new meta-heuristic algorithms have been proposed for solving optimization problems. According to the Non-Free Lunch theorem, we learn that no single algorithm can solve all optimization problems. In order to solve industrial engineering design problems more efficiently, we, inspired by the algorithm framework of the Arithmetic Optimization Algorithm (AOA) and the Harris Hawks Optimization (HHO), propose a novel hybrid algorithm based on these two algorithms, named EAOAHHO in this paper. The pinhole imaging opposition-based learning is introduced into the proposed algorithm to increase the original population diversity and the capability to escape from local optima. Furthermore, the introduction of composite mutation strategy enhances the proposed EAOAHHO exploitation and exploration to obtain better convergence accuracy. The performance of EAOAHHO is verified on 23 benchmark functions and the IEEE CEC2017 test suite. Finally, we verify the superiority of the proposed EAOAHHO over the other advanced meta-heuristic algorithms for solving four industrial engineering design problems.
Witnessing quantum steering by means of the Fisher information

Capturing specific kinds of quantum correlation is of paramount importance for quantum networking. Different routes can be taken to achieve this task, highlighting different aspects of such quantum correlations. Following the recent theoretical results by Yadin, Fadel, and Gessner [Nat. Commun. 12, 2410 (2021)], we demonstrate experimentally how steering manifests in the metrological abilities of a bipartite state. Our results confirm the relevance of this approach, and compare the outcome with already employed alternatives.
Rethinking Hard-Parameter Sharing in Multi-Domain Learning

Hard parameter sharing in multi-domain learning (MDL) al-lows domains to share some of the model parameters to reduce storage cost while improving prediction accuracy. One com-mon sharing practice is to share the bottom layers of a deep neural network among domains while using separate top layers for each domain. In this work, we revisit this common practice via an empirical study on image classification tasks from a diverse set of visual domains and make two surprising obser-vations. (1) Using separate bottom-layer parameters could achieve significantly better performance than the common practice and this phenomenon holds with different experimen-tal settings. (2) A multi-domain model with a small proportion of domain-specific parameters from bottom layers can achieve competitive performance with independent models trained on each domain separately. Our observations suggest that people adopt the new strategy of using separate bottom-layer parame-ters as a stronger baseline for model design in MDL.
Generate, Annotate, and Learn: NLP with Synthetic Text

Abstract This paper studies the use of language models as a source of synthetic unlabeled text for NLP. We formulate a general framework called “generate, annotate, and learn (GAL)” to take advantage of synthetic text within knowledge distillation, self-training, and few-shot learning applications. To generate high-quality task-specific text, we either fine-tune LMs on inputs from the task of interest, or prompt large LMs with few examples. We use the best available classifier to annotate synthetic text with soft pseudo labels for knowledge distillation and self-training, and use LMs to obtain hard labels for few-shot learning. We train new supervised models on the combination of labeled and pseudo-labeled data, which results in significant gains across several applications. We investigate key components of GAL and present theoretical and empirical arguments against the use of class-conditional LMs to generate synthetic labeled text instead of unlabeled text. GAL achieves new state-of-the-art knowledge distillation results for 6-layer transformers on the GLUE leaderboard.
Improving Calibration for Long-Tailed Recognition

Deep neural networks may perform poorly when training datasets are heavily class-imbalanced. Recently, two-stage methods decouple representation learning and classifier learning to improve performance. But there is still the vital issue of miscalibration. To address it, we design two methods to improve calibration and performance in such scenarios. Motivated by the fact that predicted probability distributions of classes are highly related to the numbers of class instances, we propose label-aware smoothing to deal with different degrees of over-confidence for classes and improve classifier learning. For dataset bias between these two stages due to different samplers, we further propose shifted batch normalization in the decoupling framework. Our proposed methods set new records on multiple popular long-tailed recognition benchmark datasets, including CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, Places-LT, and iNaturalist 2018. Code will be available at https://github.com/Jia-Research-Lab/MiSLAS.
STMTrack: Template-free Visual Tracking with Space-time Memory Networks

Boosting performance of the offline trained siamese trackers is getting harder nowadays since the fixed information of the template cropped from the first frame has been almost thoroughly mined, but they are poorly capable of resisting target appearance changes. Existing trackers with template updating mechanisms rely on time-consuming numerical optimization and complex hand-designed strategies to achieve competitive performance, hindering them from real-time tracking and practical applications. In this paper, we propose a novel tracking framework built on top of a space-time memory network that is competent to make full use of historical information related to the target for better adapting to appearance variations during tracking. Specifically, a novel memory mechanism is introduced, which stores the historical information of the target to guide the tracker to focus on the most informative regions in the current frame. Furthermore, the pixel-level similarity computation of the memory network enables our tracker to generate much more accurate bounding boxes of the target. Extensive experiments and comparisons with many competitive trackers on challenging large-scale benchmarks, OTB-2015, TrackingNet, GOT-10k, LaSOT, UAV123, and VOT2018, show that, without bells and whistles, our tracker outperforms all previous state-of-the-art real-time methods while running at 37 FPS. The code is available at https://github.com/fzh0917/STMTrack.
Inpainting Digital Dunhuang Murals with Structure-Guided Deep Network

Inpainting deteriorated regions in digital Dunhuang murals is important for Dunhuang mural content preservation. Algorithms of mural image inpainting help simplify the digital restoration process of the deteriorated murals. Most of the existing algorithms can restore plausible content for homogeneous missing regions in Dunhuang mural images. However, they often fail to fill accurate color in missing regions that contain complex structures, which is mainly due to the neglect of color relevance between positions in the missing structural region and the non-missing color regions. In this article, we propose a deep learning–based, structure-guided inpainting method for the Dunhuang mural image, which utilizes relevant color information in deep features to improve the color inpainting quality for structural regions. Specifically, we design a structure-guided feature refinement module, which explicitly leverages color relevance implied in structure information to select relevant features for refining features in the missing region. In addition, we propose a multi-step scheme for feature refinement to better propagate non-missing region feature information to the missing region. We conduct experiments on Dunhuang660 and Dunhuang No. 7 Grotto datasets. The results demonstrate that our proposed method can achieve improved color inpainting quality for missing structural regions in Dunhuang mural images.
Investigating Natural Language Processing Techniques for a Recommendation System to Support Employers, Job Seekers and Educational Institutions

Skills are the common ground between employers, job seekers and educational institutions which can be analyzed with the help of natural language processing (NLP) techniques. In this paper we explore a state-of-the-art pipeline that extracts, vectorizes, clusters, and compares skills to provide recommendations for all three parties—thereby bridging the gap between employers, job seekers and educational institutions. Our best system combines Sentence-BERT [1], UMAP [2], DBSCAN [3], and K-means clustering [4].
Robust Text CAPTCHAs Using Adversarial Examples

CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) is a widely used technology to distinguish real users and automated users such as bots. However, the advance of AI technologies weakens many CAPTCHA tests and can induce security concerns. In this paper, we propose a user-friendly text-based CAPTCHA generation method named Robust Text CAPTCHA (RTC). At the first stage, the foregrounds and backgrounds are constructed with font and background images respectively sampled from font and image libraries, and they are then synthesized into identifiable pseudo adversarial CAPTCHAs. At the second stage, we utilize a highly transferable adversarial attack designed for text CAPTCHAs to better obstruct CAPTCHA solvers. Our experiments cover comprehensive models including shallow models such as KNN, SVM and random forest, as well as various deep neural networks and OCR models. Experiments show that our CAPTCHAs have a failure rate lower than one millionth in general and high usability. They are also robust against various defensive techniques that attackers may employ, including adversarially trained CAPTCHA solvers and solvers trained with collected RTCs using manual annotation. Codes available at https://github.com/RulinShao/RTC.
Development of a Scale to Measure Cybercrime-Awareness on Social Media

This study developed a psychometric scale to measure users' cybercrime awareness level on social media. Psychometric properties of the Cybercrime Awareness on Social Media Scale (CASM-S) were tested based on data collected from 1045 social media users. Exploratory factor analysis (EFA) with principal components analysis was used to identify the underlying factor structure of the scale (N = 545). The results revealed that the scale has a unidimensional factor structure. The scale was found to have a high internal reliability (α = .957). Confirmatory factor analysis (CFA) was conducted to verify factor structure of the CASM-S (N = 500). Results revealed that the one-factor model fits the data well (x2/DF = 2.757, CFI = .939, SRMR = .0366, RMSEA = .059). Further, the study evaluated the concurrent validity of the scale (r = .855, p < .001). The findings revealed that the CASM-S is a reliable and valid tool to measure users' cybercrime awareness level on social media.