Reviewerly/exHarmony huggingface dataset:
Deep Reasoning Network for Few-shot Semantic Segmentation

Few-shot Semantic Segmentation (FSS) is a challenging problem in computer vision. It aims at segmenting objects of the unseen categories given only one or several annotated samples. The essence of FSS is to disseminate information from support images to query images for segmenting the mutual object categories. In this paper, we propose a Dynamic Reasoning Network (DRNet) to adaptively generate the parameters of predicting layers and infer the segmentation mask for each unseen category. More specifically, an Attentional Feature Integration Sub-network (AFIS) is first proposed to extract consistent features from support im-ages and query images. With shared weights, it stimulates the category consistency of different data streams. Then a Pooling-based Guidance Module (PGM) is used to cor-relate support features with query features progressively. To disseminate information from support images to various query images, we further propose a Dynamic PredictionModule (DPM) for generating the parameters of predicting layers. The proposed modules are unified for the dynamic reasoning of each query image segmentation. Experiments on two public benchmarks have demonstrated that our approach achieves superior performance and outperforms thevery recent state-of-the-art methods.
Movie Recommendation Systems Using Actor-Based Matrix Computations in South Korea

A recommendation system saves the user the trouble of searching for information and analyzes their profile to recommend the most suitable content. A variety of techniques, including content-based, collaborative, and knowledge-based techniques, have been proposed for performing recommendations. Recommendation systems are employed to suggest content, such as books, music, and videos; furthermore, they are often used in e-commerce. In particular, the South Korean film industry recommends movies using a collaborative filtering method based on genres, which is commonly used in film recommendation systems. However, this method can be ineffective when users initially encounter film recommendation services or have specific preferences with respect to movies, such as preferences regarding actors or directors. This motivated us to propose an actor-based recommendation system using the content-based filtering that considers actors’ filmography information and the genres of 509 South Korean movies. The effectiveness and performance of the suggested system are evaluated in comparison with those of a traditional genre-oriented recommendation system.
Anti-Distillation Backdoor Attacks: Backdoors Can Really Survive in Knowledge Distillation

Motivated by resource-limited scenarios, knowledge distillation (KD) has received growing attention, effectively and quickly producing lightweight yet high-performance student models by transferring the dark knowledge from large teacher models. However, many pre-trained teacher models are downloaded from public platforms that lack necessary vetting, posing a possible threat to knowledge distillation tasks. Unfortunately, thus far, there has been little research to consider the backdoor attack from the teacher model into student models in KD, which may pose a severe threat to its wide use. In this paper, we, for the first time, propose a novel Anti-Distillation Backdoor Attack (ADBA), in which the backdoor embedded in the public teacher model can survive the knowledge distillation process and thus be transferred to secret distilled student models. We first introduce a shadow to imitate the distillation process and adopt an optimizable trigger to transfer information to help craft the desired teacher model. Our attack is powerful and effective, which achieves 95.92%, 94.79%, and 90.19% average success rates of attacks (SRoAs) against several different structure student models on MNIST, CIFAR-10, and GTSRB, respectively. Our ADBA also performs robustly under different user distillation environments with 91.72% and 92.37% average SRoAs on MNIST and CIFAR-10, respectively. Finally, we show that the ADBA has a low overhead in the injecting process, which converges on 50 and 70 epochs on CIFAR-10 and GTSRB, respectively, while the normal training epochs of these datasets are almost 200.
Entanglement structures in quantum field theories: Negativity cores and bound entanglement in the vacuum

The many-body entanglement between two finite (size-$d$) disjoint vacuum regions of noninteracting lattice scalar field theory in one spatial dimension, i.e., a ${({d}_{A}\ifmmode\times\else\texttimes\fi{}{d}_{B})}_{\mathrm{mixed}}$ Gaussian continuous variable system, is locally transformed into a tensor-product core of ${({1}_{A}\ifmmode\times\else\texttimes\fi{}{1}_{B})}_{\mathrm{mixed}}$ entangled pairs. Accessible entanglement within these core pairs exhibits an exponential hierarchy and as such identifies the structure of dominant region modes from which vacuum entanglement could be extracted into a spatially separated pair of quantum detectors. Beyond the core, the remaining modes of the halo are determined to be $AB$ separable in isolation, as well as separable from the core. However, state preparation protocols that distribute entanglement in the form of ${({1}_{A}\ifmmode\times\else\texttimes\fi{}{1}_{B})}_{\mathrm{mixed}}$ core pairs are found to require additional entanglement in the halo that is obscured by classical correlations. This inaccessible (bound) halo entanglement is found to mirror the accessible entanglement, but with a step behavior as the continuum is approached. It remains possible that alternate initialization protocols that do not utilize the exponential hierarchy of core-pair entanglement may require less inaccessible entanglement. Entanglement consolidation is expected to persist in higher dimensions and may aid classical and quantum simulations of asymptotically free gauge field theories, such as quantum chromodynamics.
High Frequency Detail Accentuation in CNN Image Restoration

Given its nature of statistical inference, machine learning methods incline to downplay relatively rare events. But in many applications statistical outliers carry disproportional significance; they can, if being left without special treatment as of now, cause CNNs to perform unsatisfactorily on instances of interests. This is the reason why existing CNN image restoration methods all suffer from the problem of blurred details. To overcome this weakness, we advocate a new training methodology to sensitize the CNNs to desired events even they are atypical. Specifically for image restoration, we propose a so-called high frequency feature accentuation space that promotes image sharpness and clarity by maximally discriminating the ground truth image and the CNN-restored image in atypical but semantically important features. Then we force the restored image to agree with the ground truth image in the feature accentuation space by including an auxiliary loss term in the training process. This aims at a high degree of agreement of the two images on high frequency constructs such as sharp edges and fine textures, i.e., penalizes image blurs. The new CNN design method is implemented and tested for tasks of image super-resolution and denoising. Experimental results demonstrate the achievement of our design objective.
Teaching Innovation Experience for COVID-19 Times: A Case Study on Blended Learning of Television Journalism Courses with Moodle

The effective implementation of information and communications technologies (ICTs) in higher education is not guaranteed without serious and rigorous pedagogical reflection. It is essential to maintain an ongoing debate on the effectiveness of the learning process. A debate accelerated by the impact of the COVID-19 crisis and the growing role of virtual and remote learning in universities worldwide. This research aims to overcome the quantitative approach of previous work with qualitative evidence to contribute to the field knowledge. The designed case study is based on a Moodle teaching experience in communication studies at a Spanish university. This innovative methodology is developed within a hybrid teaching ecosystem (blended learning) with the support of a virtual platform (Aula Virtual). The sample analysed is composed of 77 students enrolled in the subject ‘Television Journalism’. The design of an original and novel data collection instrument has supplied various quantitative and qualitative data. The triangulation of four data collection techniques has provided a demographic profile, an activity report, a grading report and a satisfaction report. The findings suggest a high degree of student satisfaction, with the experience and a significant improvement in students’ television writing skills, thanks to the implementation of Moodle.
Gaussian Kernel Based SVR Model for Short-Term Photovoltaic MPP Power Prediction

Predicting the power obtained at the output of the photovoltaic (PV) system is fundamental for the optimum use of the PV system. However, it varies at different times of the day depending on intermittent and nonlinear environmental conditions including solar irradiation, temperature and the wind speed, Short-term power prediction is vital in PV systems to reconcile generation and demand in terms of the cost and capacity of the reserve. In this study, a Gaussian kernel based Support Vector Regression (SVR) prediction model using multiple input variables is proposed for estimating the maximum power obtained from using perturb observation method in the different irradiation and the different temperatures for a short-term in the DC-DC boost converter at the PV system. The performance of the kernel-based prediction model depends on the availability of a suitable kernel function that matches the learning objective, since an unsuitable kernel function or hyper parameter tuning results in significantly poor performance. In this study for the first time in the literature both maximum power is obtained at maximum power point and short-term maximum power estimation is made. While evaluating the performance of the suggested model, the PV power data simulated at variable irradiations and variable temperatures for one day in the PV system simulated in MATLAB were used. The maximum power obtained from the simulated system at maximum irradiance was 852.6 W. The accuracy and the performance evaluation of suggested forecasting model were identified utilizing the computing error statistics such as root mean square error (RMSE) and mean square error (MSE) values. MSE and RMSE rates which obtained were 4.5566 * 10−04 and 0.0213 using ANN model. MSE and RMSE rates which obtained were 13.0000 * 10−04 and 0.0362 using SWD-FFNN model. Using SVR model, 1.1548 * 10−05 MSE and 0.0034 RMSE rates were obtained. In the short-term maximum power prediction, SVR gave higher prediction performance according to ANN and SWD-FFNN.
Robust Real-World Image Super-Resolution against Adversarial Attacks

Recently deep neural networks (DNNs) have achieved significant success in real-world image super-resolution (SR). However, adversarial image samples with quasi-imperceptible noises could threaten deep learning SR models. In this paper, we propose a robust deep learning framework for real-world SR that randomly erases potential adversarial noises in the frequency domain of input images or features. The rationale is that on the SR task clean images or features have a different pattern from the attacked ones in the frequency domain. Observing that existing adversarial attacks usually add high-frequency noises to input images, we introduce a novel random frequency mask module that blocks out high-frequency components possibly containing the harmful perturbations in a stochastic manner. Since the frequency masking may not only destroys the adversarial perturbations but also affects the sharp details in a clean image, we further develop an adversarial sample classifier based on the frequency domain of images to determine if applying the proposed mask module. Based on the above ideas, we devise a novel real-world image SR framework that combines the proposed frequency mask modules and the proposed adversarial classifier with an existing super-resolution backbone network. Experiments show that our proposed method is more insensitive to adversarial attacks and presents more stable SR results than existing models and defenses.
Interactive Labeling for Human Pose Estimation in Surveillance Videos

Automatically detecting and estimating the movement of persons in real-world uncooperative scenarios is very challenging in great part due to limited and unreliably annotated data. For instance annotating a single human body pose for activity recognition requires 40-60 seconds in complex sequences, leading to long-winded and costly annotation processes. Therefore increasing the sizes of annotated datasets through crowdsourcing or automated annotation is often used at a great financial costs, without reliable validation processes and inadequate annotation tools greatly impacting the annotation quality. In this work we combine multiple techniques into a single web-based general-purpose annotation application. Pre-trained machine learning models enable annotators to interactively detect pedestrians, re-identify them throughout the sequence, estimate their poses, and correct annotation suggestions in the same interface. Annotations are then inter- and extrapolated between frames. The application is evaluated through several user studies and the results are extensively analyzed. Experiments demonstrate a 55% reduction in annotation time for less complex scenarios while simultaneously decreasing perceived annotator workload.
MedSkip: Medical Report Generation Using Skip Connections and Integrated Attention

Medical scans are extremely important for accurate diagnosis and treatment. To assist staff members in such crucial tasks, developing a computer vision model that efficiently processes a medical image and results in a generated report can be highly beneficial. Such a robust system can not only act as a helping hand for professionals but also eliminate the chances of error that might arise in the case of in-experienced staff members. However, previous studies lack focus on experimenting with the visual extractor, which is of eminent importance. Keeping this in mind, we propose a novel architecture of a modified HRNet which includes added skip connections along with convolutional block attention modules (CBAM). The entire architecture can be divided into two components, the first being the visual extractor where the pre-processed image is fed into the HRNet convolutional layers. Outputs of each down-sampled layer are concatenated after passing through the attention modules. The second component includes the use of a memory-driven Transformer that generates the report. We evaluate our model on two publicly available datasets, PEIR Gross and IU X-Ray, establishing new state-of-the-art for PEIR Gross while giving competitive results for IU X-Ray.
Learning Hierarchal Channel Attention for Fine-grained Visual Classification

Learning delicate feature representation of object parts plays a critical role in fine-grained visual classification tasks. However, advanced deep convolutional neural networks trained for general visual classification tasks usually tend to focus on the coarse-grained information while ignoring the fine-grained one, which is of great significance for learning discriminative representation. In this work, we explore the great merit of multi-modal data in introducing semantic knowledge and sequential analysis techniques in learning hierarchical feature representation for generating discriminative fine-grained features. To this end, we propose a novel approach, termed Channel Cusum Attention ResNet (CCA-ResNet ), for multi-modal joint learning of fine-grained representation. Specifically, we use feature-level multi-modal alignment to connect image and text classification models for joint multi-modal training. Through joint training, image classification models trained with semantic level labels tend to focus on the most discriminative parts, which enhances the cognitive ability of the model. Then, we propose a Channel Cusum Attention (CCA ) mechanism to equip feature maps with hierarchical properties through unsupervised reconstruction of local and global features. The benefits brought by the CCA are in two folds: a) allowing fine-grained features from early layers to be preserved in the forward propagation of deep networks; b) leveraging the hierarchical properties to facilitate multi-modal feature alignment. We conduct extensive experiments to verify that our proposed model can achieve state-of-the-art performance on a series of fine-grained visual classification benchmarks.
Unauthorized AI cannot recognize me: Reversible adversarial example

In this study, we propose a new methodology to control how user’s data is recognized and used by AI via exploiting the properties of adversarial examples. For this purpose, we propose reversible adversarial example (RAE), a new type of adversarial example. A remarkable feature of RAE is that the image can be correctly recognized and used by the AI model specified by the user because the authorized AI can recover the original image from the RAE exactly by eliminating adversarial perturbation. On the other hand, other unauthorized AI models cannot recognize it correctly because it functions as an adversarial example. Moreover, RAE can be considered as one type of encryption to computer vision since reversibility guarantees the decryption. To realize RAE, we combine three technologies, adversarial example, reversible data hiding for exact recovery of adversarial perturbation, and encryption for selective control of AIs who can remove adversarial perturbation. Experimental results show that the proposed method can achieve comparable attack ability with the corresponding adversarial attack method and similar visual quality with the original image, including white-box attacks and black-box attacks.
Prediction of Interlocutors’ Subjective Impressions Based on Functional Head-Movement Features in Group Meetings

A novel model is proposed to predict interlocutors’ subjective impressions during group meetings based on their head movements. The goal is to explore the potential of the communicative functions performed through head movements. To this end, this paper focuses on ten frequent functions, such as speaker emphasis and listener back-channel responses, which are detected using convolutional neural networks (CNNs) from head-pose sequences. Regarding the prediction target, this study employs four items of subjective impressions—atmosphere, enjoyment, willingness, and concentration—which are scored at every two-minute interval by the interlocutors themselves in four-party meetings with 17 groups. From the detected head-movement functions, this paper newly defines the functional features, including function occurrence rates and composition ratios, in addition to the kinetic features representing head-movement activity. Using these features as the input, random forest regressors predict the impression scores. Compared with the baseline model using only kinetic features, the prediction model using both kinetic and functional features improved prediction performance. The percentage of moderate or higher correlations (≥ 0.5) between reported scores and predictions increased from 25% to 41% in all groups and all items. These results suggest that head-movement functions could be effective cues for predicting interlocutors’ subjective impressions.
Counterfactual Debiasing Inference for Compositional Action Recognition

Compositional action recognition is a novel challenge in the computer vision community and focuses on revealing the different combinations of verbs and nouns instead of treating subject-object interactions in videos as individual instances only. Existing methods tackle this challenging task by simply ignoring appearance information or fusing object appearances with dynamic instance tracklets. However, those strategies usually do not perform well for unseen action instances. For that, in this work we propose a novel learning framework called Counterfactual Debiasing Network (CDN) to improve the model generalization ability by removing the interference introduced by visual appearances of objects/subjects. It explicitly learns the appearance information in action representations and later removes the effect of such information in a causal inference manner. Specifically, we use tracklets and video content to model the factual inference by considering both appearance information and structure information. In contrast, only video content with appearance information is leveraged in the counterfactual inference. With the two inferences, we conduct a causal graph which captures and removes the bias introduced by the appearance information by subtracting the result of the counterfactual inference from that of the factual inference. By doing that, our proposed CDN method can better recognize unseen action instances by debiasing the effect of appearances. Extensive experiments on the Something-Else dataset clearly show the effectiveness of our proposed CDN over existing state-of-the-art methods.
Unsupervised Generative Adversarial Networks with Cross-model Weight Transfer Mechanism for Image-to-image Translation

Image-to-image translation covers a variety of application scenarios in reality, and is one of the key research directions in computer vision. However, due to the defects of GAN, current translation frameworks may encounter model collapse and low quality of generated images. To solve the above problems, this paper proposes a new model CWT-GAN, which introduces the cross-model weight transfer mechanism. The discriminator of CWT-GAN has the same encoding module structure as the generator’s. In the training process, the discriminator will transmit the weight of its encoding module to the generator in a certain proportion after each weight update. CWT-GAN can generate diverse and higher-quality images with the aid of the weight transfer mechanism, since features learned by discriminator tend to be more expressive than those learned by generator trained via maximum likelihood. Extensive experiments demonstrate that our CWT-GAN performs better than the state-of-the-art methods in a single translation direction for several datasets.
CG-GAN

Old photos are an important carrier to preserve the past. Usually, the degradation of old photos is rather diverse and complex. Therefore, the existing methods to solve conventional restoration tasks are difficult to generalize. To solve this problem, we propose a novel method based on generative adversarial network. Our method utilizes the class-attributes of old photos to complete restoration in latent space. Specifically, we divide the process of restoring old photos into two stages, one is global defect restoration stage and the other is local detail restoration stage. In global defect restoration stage, we extract the latent representations of four classes of high-level attributes that are smoothness, clarity, connectivity and completeness. We use latent class-attribute information to restore global defects in latent space and we obtain conditional control vector through a condition network to guide the subsequent local detail restoration stage. In local detail restoration stage, we propose a dynamic condition-guided restoration module that selects the most suitable combination of features to further restore local details through a dynamic network. In addition, we propose a dual discriminator to pay more attention to style and defect restoration. We ignore the complex degradation of old photos to directly restore advanced class-attributes. Therefore, our method has better generalization performance. Experiments show that our method is superior to other existing methods of image restoration in terms of visual quality and numerical metrics.