Reviewerly/exHarmony huggingface dataset:
Weakly Supervised Real-time Image Cropping based on Aesthetic Distributions

Image cropping is an effective tool to edit and manipulate images to achieve better aesthetic quality. Most existing cropping approaches rely on the two-step paradigm where multiple candidate cropping areas are proposed initially and the optimal cropping window is determined based on some quality criteria for these candidates afterwards. The obvious disadvantage of this mechanism is its low efficiency due to the huge searching space of candidate crops. In order to tackle this problem, a weakly supervised cropping framework is proposed, where the distribution dissimilarity between high quality images and cropped images is used to guide the coordinate predictor's training and the ground truths of cropping windows are not required by the proposed method. Meanwhile, to improve the cropping performance, a saliency loss is also designed in the proposed framework to force the neural network to focus more on the interested objects in the image. Under this framework, the images can be cropped effectively by the trained coordinate predictor in a one-pass favor without multiple candidates proposals, which ensures the high efficiency of the proposed system . Also, based on the proposed framework, many existing distribution dissimilarity measurements can be applied to train the image cropping system with high flexibility, such as likelihood based and divergence based distribution dissimilarity measure proposed in this work. The experiments on the public databases show that the proposed cropping method achieves the state-of-the-art accuracy, and the high computation efficiency as fast as 285 FPS is also obtained.
Video Synthesis via Transform-Based Tensor Neural Network

Video frame synthesis is an important task in computer vision and has drawn great interests in wide applications. However, existing neural network methods do not explicitly impose tensor low-rankness of videos to capture the spatiotemporal correlations in a high-dimensional space, while existing iterative algorithms require hand-crafted parameters and take relatively long running time. In this paper, we propose a novel multi-phase deep neural network Transform-Based Tensor-Net that exploits the low-rank structure of video data in a learned transform domain, which unfolds an Iterative Shrinkage-Thresholding Algorithm (ISTA) for tensor signal recovery. Our design is based on two observations: (i) both linear and nonlinear transforms can be implemented by a neural network layer, and (ii) the soft-thresholding operator corresponds to an activation function. Further, such an unfolding design is able to achieve nearly real-time at the cost of training time and enjoys an interpretable nature as a byproduct. Experimental results on the KTH and UCF-101 datasets show that compared with the state-of-the-art methods, i.e., DVF and Super SloMo, the proposed scheme improves Peak Signal-to-Noise Ratio (PSNR) of video interpolation and prediction by 4.13 dB and 4.26 dB, respectively.
Privacy-preserving fully anonymous ciphertext policy attribute-based broadcast encryption with constant-size secret keys and fast decryption

Cloud computing is admired as a notable computing terminology in which facilities of the computing framework are rendered as aid over Internet services. In the era of cloud computing, lightweight devices can be used to afford remote e-healthcare, which facilitates the development of remote healthcare consultations, monitoring of patients’ health reports, and medical research. The downside of the cloud environment is that it does not support direct revocation with fine-grained access control over encrypted data by specifying access policies among ciphertexts and consumers’ secret-keys. Ciphertext Policy Attribute-Based Broadcast Encryption (CP-ABBE) can be deployed in a cloud computing environment to ensure fine-grained access control of outsourced data. In a CP-ABBE, consumers’ secret-keys are associated with attributes. Broadcaster encrypts the data concerning a subscriber set and an access policy. Legitimate consumers can recover the correct data if and only if the data file access policy satisfies the attributes associated with the consumers’ secret-keys. However, existing CP-ABBE schemes fail to withstand social engineering attacks, meaning that those are incapable of accomplishing consumers’ full anonymity as well as privacy of access policy, which are highly sensitive information in e-healthcare scenario. To mitigate these deficiencies, we introduce a new CP-ABBE, called privacy-preserving fully anonymous CP-ABBE, by tweaking an identity-based encryption framework over the most advanced and secure Type-3 bilinear pairings. In addition to being adaptively secure, our construction endures indistinguishable chosen-plaintext attack security under standard asymmetric decisional bilinear Diffie–Hellman assumption without using the random oracle model. In particular, the design is very efficient both in terms of parameter sizes and computation costs, as consumer secret-key and decryption time both are constant, and ciphertext-size is independent of access policy. The implementation and performance analysis of the protocol in opposition to existing works illustrate its effectiveness in providing healthcare services in the cloud environment.
DiPLIP: Distributed Parallel Processing Platform for Stream Image Processing Based on Deep Learning Model Inference

Recently, as the amount of real-time video streaming data has increased, distributed parallel processing systems have rapidly evolved to process large-scale data. In addition, with an increase in the scale of computing resources constituting the distributed parallel processing system, the orchestration of technology has become crucial for proper management of computing resources, in terms of allocating computing resources, setting up a programming environment, and deploying user applications. In this paper, we present a new distributed parallel processing platform for real-time large-scale image processing based on deep learning model inference, called DiPLIP. It provides a scheme for large-scale real-time image inference using buffer layer and a scalable parallel processing environment according to the size of the stream image. It allows users to easily process trained deep learning models for processing real-time images in a distributed parallel processing environment at high speeds, through the distribution of the virtual machine container.
Hyperparameter Auto-Tuning in Self-Supervised Robotic Learning

Policy optimization in reinforcement learning requires the selection of numerous hyperparameters across different environments. Fixing them incorrectly may negatively impact optimization performance leading notably to insufficient or redundant learning. Insufficient learning (due to convergence to local optima) results in under-performing policies whilst redundant learning wastes time and resources. The effects are further exacerbated when using single policies to solve multi-task learning problems. Observing that the Evidence Lower Bound (ELBO) used in Variational Auto-Encoders correlates with the diversity of image samples, we propose an auto-tuning technique based on the ELBO for self-supervised reinforcement learning. Our approach can auto-tune three hyperparameters: the replay buffer size, the number of policy gradient updates during each epoch, and the number of exploration steps during each epoch. We use a state-of-the-art self-supervised robot learning framework (Reinforcement Learning with Imagined Goals (RIG) using Soft Actor-Critic) as baseline for experimental verification. Experiments show that our method can auto-tune online and yields the best performance at a fraction of the time and computational resources. Code, video, and appendix for simulated and real-robot experiments can be found at the project page \url{www.JuanRojas.net/autotune}.
Beyond the Attention: Distinguish the Discriminative and Confusable Features For Fine-grained Image Classification

Learning subtle discriminative features plays a significant role in fine-grained image classification. Existing methods usually extract the distinguishable parts through the attention module for classification. Although these learned distinguishable parts contain valuable features that are beneficial for classification, part of irrelevant features are also preserved, which may confuse the model to make a correct classification, especially for the fine-grained tasks due to their similarities. How to keep the discriminative features while removing confusable features from the distinguishable parts is an interesting yet changeling task. In this paper, we introduce a novel classification approach, named Logical-based Feature Extraction Model (LAFE for short) to address this issue. The main advantage of LAFE lies in the fact that it can explicitly add the significance of discriminative features and subtract the confusable features. Specifically, LAFE utilizes the region attention modules and channel attention modules to extract discriminative features and confusable features respectively. Based on this, two novel loss functions are designed to automatically induce attention over these features for fine-grained image classification. Our approach demonstrates its robustness, efficiency, and state-of-the-art performance on three benchmark datasets.
ThumbNet: One Thumbnail Image Contains All You Need for Recognition

Although deep convolutional neural networks (CNNs) have achieved great success in computer vision tasks, its real-world application is still impeded by its voracious demand of computational resources. Current works mostly seek to compress the network by reducing its parameters or parameter-incurred computation, neglecting the influence of the input image on the system complexity. Based on the fact that input images of a CNN contain substantial redundancy, in this paper, we propose a unified framework, dubbed as ThumbNet, to simultaneously accelerate and compress CNN models by enabling them to infer on one thumbnail image. We provide three effective strategies to train ThumbNet. In doing so, ThumbNet learns an inference network that performs equally well on small images as the original-input network on large images. With ThumbNet, not only do we obtain the thumbnail-input inference network that can drastically reduce computation and memory requirements, but also we obtain an image downscaler that can generate thumbnail images for generic classification tasks. Extensive experiments show the effectiveness of ThumbNet, and demonstrate that the thumbnail-input inference network learned by ThumbNet can adequately retain the accuracy of the original-input network even when the input images are downscaled 16 times.
A Comparative Usability Study of Bare Hand Three-Dimensional Object Selection Techniques in Virtual Environment

Object selection is the basis of natural user–computer interaction (NUI) in a virtual environment (VE). Among the three-dimensional object selection techniques employed in virtual reality (VR), bare hand-based finger clicking interaction and ray-casting are two convenient approaches with a high level of acceptance. This study involved 14 participants, constructed a virtual laboratory environment in VR, and compared the above two finger-based interaction techniques in terms of aspects of the task performance, including the success rate, total reaction time, operational deviation, and accuracy, at different spatial positions. The results indicated that the applicable distance range of finger clicking interaction and finger ray-casting was 0.2 to 1.4 m and over 0.4 m, respectively. Within the shared applicable distance, the finger clicking interaction achieved a shorter total reaction time and higher clicking accuracy. The performance of finger clicking interaction varied remarkably at the center and edge of the horizontal field of view, while no significant difference was found among ray-casting at various horizontal azimuths. The current findings could be directly applied to the application of bare-hand interaction in VR environments.
National Research and Federal Universities Contribution to the Project 5-100

This article presents research results of national research universities (NRU) and federal universities (FU) contribution to the Project of competitiveness enhancement of the leading Russian universities among global research and education centers (Project 5-100). The following indicators were analyzed: publications, indexed in Web of Science and Scopus databases, share of international students and faculty, share of young employees and staff with PhD, performance in international rankings, etc. The analysis was based on 41 quantitative and 11 qualitative indicators. In the analysis the following data sources were used: statistical forms 1-Monitoring, rankings agencies data, etc. It has been determined that the most influence on the Russian education competitiveness enhancement in the world belongs to the leading NRU – participants of Project 5-100. The article analyzes the contribution of federal universities in the implementation of the Project 5-100.
Worldwide Daily Optimum Tilt Angle Model to Obtain Maximum Solar Energy

This paper aims to obtain the overall equations of the yearly, seasonal, monthly and daily optimum tilt angles worldwide. So, the daily optimum tilt angle variations are examined for 337 locations in the northern hemisphere and 247 locations in the southern hemisphere, and it is shown that the daily optimum tilt angle changes in cosine form over a year. Then, the daily optimum tilt angle is estimated using a cosine function. The daily optimum tilt angle is independent of the longitude and can be estimated using a function that depends on the latitude and the number of days. The resulting model can be used to calculate the optimum tilt angles in desired intervals. In order to verify the validity of the results, the results of the proposed model are compared with the results of the other papers. It is shown that the presented model can calculate the optimum tilt angles with acceptable accuracy in the southern and northern hemispheres.
Assessing Mathematics Misunderstandings via Bayesian Inverse Planning

Abstract Online educational technologies offer opportunities for providing individualized feedback and detailed profiles of students' skills. Yet many technologies for mathematics education assess students based only on the correctness of either their final answers or responses to individual steps. In contrast, examining the choices students make for how to solve the equation and the ways in which they might answer incorrectly offers the opportunity to obtain a more nuanced perspective of their algebra skills. To automatically make sense of step‐by‐step solutions, we propose a Bayesian inverse planning model for equation solving that computes an assessment of a learner's skills based on her pattern of errors in individual steps and her choices about what sequence of problem‐solving steps to take. Bayesian inverse planning builds on existing machine learning tools to create a generative model relating (mis)‐understandings to equation solving choices. Two behavioral experiments demonstrate that the model can interpret people's equation solving and that its assessments are consistent with those of experienced teachers. A third experiment uses this model to tailor guidance for learners based on individual differences in misunderstandings, closing the loop between assessing understanding, and using that assessment within an educational technology. Finally, because the bottleneck in applying inverse planning to a new domain is in creating the model of possible student misunderstandings, we show how to combine inverse planning with an existing production rule model to make inferences about student misunderstandings of fraction arithmetic.
A Visual Inductive Priors Framework for Data-Efficient Image Classification

State-of-the-art classifiers rely heavily on large-scale datasets, such as ImageNet, JFT-300M, MSCOCO, Open Images, etc. Besides, the performance may decrease significantly because of insufficient learning on a handful of samples. We present Visual Inductive Priors Framework (VIPF), a framework that can learn classifiers from scratch. VIPF can maximize the effectiveness of limited data. In this work, we propose a novel neural network architecture: DSK-net, which is very effective in training from small data sets. With more discriminative feature extracted from DSK-net, overfitting of network is alleviated. Furthermore, a loss function based on positive class as well as an induced hierarchy are also applied to further improve the VIPF's capability of learning from scratch. Finally, we won the 1st Place in VIPriors image classification competition.
QNet: An Adaptive Quantization Table Generator Based on Convolutional Neural Network

The JPEG is one of the most widely used lossy image-compression standards, whose compression performance depends largely on a quantization table. In this work, we utilize a Convolutional Neural Network (CNN) to generate an image-adaptive quantization table in a standard-compliant way. We first build an image set containing more than 10,000 images and generate their optimal quantization tables through a classical genetic algorithm, and then propose a method that can efficiently extract and fuse the frequency and spatial domain information of each image to train a regression network to directly generate adaptive quantization tables. In addition, we extract several representative quantization tables from the dataset and train a classification network to indicate the optimal one for each image, which further improves compression performance and computational efficiency. Tests on diverse images show that the proposed method clearly outperforms the state-of-the-art method. Compared with the standard table at the compression rate of 1.0 bpp, the regression and classification network provide average Peak Signal-to-Noise Ratio (PSNR) gains of nearly 1.2 and 1.4 dB. For the experiment under Structural Similarity Index Measurement (SSIM), the improvements are 0.4% and 0.54%, respectively. The proposed method also has competitive computational efficiency, as the regression and classification network only take 15 and 6.25 milliseconds, respectively, to process a 768 W 512 image on a single CPU core at 3.20 GHz.
Investigating the Effect of Emoji in Opinion Classification of Uzbek Movie Review Comments

Opinion mining on social media posts has become more and more popular. Users often express their opinion on a topic not only with words but they also use image symbols such as emoticons and emoji. In this paper, we investigate the effect of emoji-based features in opinion classification of Uzbek texts, and more specifically movie review comments from YouTube. Several classification algorithms are tested, and feature ranking is performed to evaluate the discriminative ability of the emoji-based features.
JointFontGAN

Automatic generation of font and text design in the wild is a challenging task since font and text in real world exhibit various visual effects. In this paper, we propose a novel model, JointFontGAN, to derive fonts, including both geometric structures and shape contents in correctness and consistency with very few font samples available. Specifically, we design an end-to-end deep learning based approach for font generation through the new multi-stream extended conditional generative adversarial network (XcGAN) models, which jointly learn and generate both font skeleton and glyph representations simultaneously. It can adapt to the geometric variability and content scalability at the neural network level. Then, we apply it, along with the developed efficient and effective one-stage model, to text generations in letters and sentences / paragraphs with both standard and artistic / handwriting styles. The extensive experiments and comparisons demonstrate that our approach outperforms the state-of-the-art methods on the collected datasets including 20K fonts (letters and punctuations) with different styles.
Transductive Multi-Object Tracking in Complex Events by Interactive Self-Training

Recently, multi-object tracking (MOT) for estimating trajectories of pedestrians has undergone fast development and played an important role in human-centric video analysis. However, video analysis in complex events (e.g. scenes in HiEve dataset) is still under-explored. In complex real-world scenarios, domain gap in unseen testing scenes and severe occlusion problem that disconnects tracks are challenging for existing online MOT methods without domain adaptation. To alleviate domain gap, we study the problem in a transductive learning setting, which assumes that unlabeled testing data is available for learning offline tracking. We propose a transductive interactive self-training method to adapt the tracking model to unseen crowded scenes with unlabeled testing data by means of teacher-student interative learning. To reduce prediction variance in an unseen domain, we train two different models and teach one model with pseudo labels of unlabeled data predicted by the other model interactively. To improve robustness against occlusions during self-training, we exploit disconnected track interpolation (DTI) to refine the predicted pseudo labels. Our method achieved MOTA of 60.23 on HiEve dataset and won the first place of Multi-person Motion Tracking in Complex Events (with Private Detection) in the ACM MM Grand Challenge on Large-scale Human-centric Video Analysis in Complex Events.
A fusion model for multi-label emotion classification based on BERT and topic clustering

As one of the most critical tasks of natural language processing (NLP), emotion classification has a wide range of applications in many fields. However, restricted by corpus, semantic ambiguity, and other constraints, researchers in emotion classification face many difficulties, and the accuracy of multi-label emotion classification is not ideal. In this paper, to improve the accuracy of multi-label emotion classification, especially when semantic ambiguity occurs, we proposed a fusion model for text based on self-attention and topic clustering. We use the Pre-trained BERT to extract the hidden emotional representations of the sentence, and use the improved LDA topic model to cluster the topics of different levels of text. Then we fuse the hidden representations of the sentence and use a classification neural network to calculate the multi-label emotional intensity of the sentence. After testing on the Chinese emotion corpus Ren_CECPs corpus, extensive experimental results demonstrate that our model outperforms several strong baselines and related works. The F1-score of our model reaches 0.484, which is 0.064 higher than the best results in similar studies.