Reviewerly/exHarmony huggingface dataset:
Detection of Manipulated Face Videos over Social Networks: A Large-Scale Study

The detection of manipulated videos represents a highly relevant problem in multimedia forensics, which has been widely investigated in the last years. However, a common trait of published studies is the fact that the forensic analysis is typically applied on data prior to their potential dissemination over the web. This work addresses the challenging scenario where manipulated videos are first shared through social media platforms and then are subject to the forensic analysis. In this context, a large scale performance evaluation has been carried out involving general purpose deep networks and state-of-the-art manipulated data, and studying different effects. Results confirm that a performance drop is observed in every case when unseen shared data are tested by networks trained on non-shared data; however, fine-tuning operations can mitigate this problem. Also, we show that the output of differently trained networks can carry useful forensic information for the identification of the specific technique used for visual manipulation, both for shared and non-shared data.
Correct Me If I am Wrong: Interactive Learning for Robotic Manipulation

Learning to solve complex manipulation tasks from visual observations is a dominant challenge for real-world robot learning. Although deep reinforcement learning algorithms have recently demonstrated impressive results in this context, they still require an impractical amount of time-consuming trial-and-error iterations. In this work, we consider the promising alternative paradigm of interactive learning in which a human teacher provides feedback to the policy during execution, as opposed to imitation learning where a pre-collected dataset of perfect demonstrations is used. Our proposed CEILing (Corrective and Evaluative Interactive Learning) framework combines both corrective and evaluative feedback from the teacher to train a stochastic policy in an asynchronous manner, and employs a dedicated mechanism to trade off human corrections with the robot’s own experience. We present results obtained with our framework in extensive simulation and real-world experiments to demonstrate that CEILing can effectively solve complex robot manipulation tasks directly from raw images in less than one hour of real-world training.
Investigating the Usability of a Head-Mounted Display Augmented Reality Device in Elementary School Children

Augmenting reality via head-mounted displays (HMD-AR) is an emerging technology in education. The interactivity provided by HMD-AR devices is particularly promising for learning, but presents a challenge to human activity recognition, especially with children. Recent technological advances regarding speech and gesture recognition concerning Microsoft's HoloLens 2 may address this prevailing issue. In a within-subjects study with 47 elementary school children (2nd to 6th grade), we examined the usability of the HoloLens 2 using a standardized tutorial on multimodal interaction in AR. The overall system usability was rated "good". However, several behavioral metrics indicated that specific interaction modes differed in their efficiency. The results are of major importance for the development of learning applications in HMD-AR as they partially deviate from previous findings. In particular, the well-functioning recognition of children's voice commands that we observed represents a novelty. Furthermore, we found different interaction preferences in HMD-AR among the children. We also found the use of HMD-AR to have a positive effect on children's activity-related achievement emotions. Overall, our findings can serve as a basis for determining general requirements, possibilities, and limitations of the implementation of educational HMD-AR environments in elementary school classrooms.
Deblur-YOLO: Real-Time Object Detection with Efficient Blind Motion Deblurring

Object detection has been a traditional yet open computer vision research field. In intensive studies, object detection models have achieved promising results regarding recognition accuracy and inference speed. However, previous state-of-the-art algorithms fail to operate at blurry images. In this work, we propose Deblur-YOLO, an efficient, YOLO-based and detection-driven approach robust to motion blur photographs. We introduce a generative adversarial network with a dilated feature pyramid generator, a pair of multi-scale discriminators with spectral normalization, and a detection discriminator. We design a new image quality metric called Smooth Peak Signal-to-Noise Ratio (SPSNR) for measuring the smoothness of the reconstructed image. Empirical studies on benchmark datasets demonstrate Deblur-YOLO's superiority. On COCO 2014, Set 5 and Setl4, Deblur-YOLO achieves leading results for parameters, deblurring time, PSNR, SPSNR and SSIM. We also visually display the excellence of our deblurring performance to competing models.
Multitask learning with single gradient step update for task balancing

Multitask learning is a methodology to boost generalization performance and also reduce computational intensity and memory usage. However, learning multiple tasks simultaneously can be more difficult than learning a single task because it can cause imbalance among tasks. To address the imbalance problem, we propose an algorithm to balance between tasks at the gradient level by applying gradient-based meta-learning to multitask learning. The proposed method trains shared layers and task-specific layers separately so that the two layers with different roles in a multitask network can be fitted to their own purposes. In particular, the shared layer that contains informative knowledge shared among tasks is trained by employing single gradient step update and inner/outer loop training to mitigate the imbalance problem at the gradient level. We apply the proposed method to various multitask computer vision problems and achieve state-of-the-art performance.
Anatomy-Constrained Contrastive Learning for Synthetic Segmentation Without Ground-Truth

A large amount of manual segmentation is typically required to train a robust segmentation network so that it can segment objects of interest in a new imaging modality. The manual efforts can be alleviated if the manual segmentation in one imaging modality (e.g., CT) can be utilized to train a segmentation network in another imaging modality (e.g., CBCT/MRI/PET). In this work, we developed an anatomy-constrained contrastive synthetic segmentation network (AccSeg-Net) to train a segmentation network for a target imaging modality without using its ground-truth. Specifically, we proposed to use anatomy-constraint and patch contrastive learning to ensure the anatomy fidelity during the unsupervised adaptation, such that the segmentation network can be trained on the adapted image with correct anatomical structure/content. The training data for our AccSeg-Net consists of 1) imaging data paired with segmentation ground-truth in source modality, and 2) unpaired source and target modality imaging data. We demonstrated successful applications on CBCT, MRI, and PET imaging data, and showed superior segmentation performances as compared to previous methods. Our code is available at https://github.com/bbbbbbzhou/AccSeg-Net
Interactive Data Comics

This paper investigates how to make data comics interactive. Data comics are an effective and versatile means for visual communication, leveraging the power of sequential narration and combined textual and visual content, while providing an overview of the storyline through panels assembled in expressive layouts. While a powerful static storytelling medium that works well on paper support, adding interactivity to data comics can enable non-linear storytelling, personalization, levels of details, explanations, and potentially enriched user experiences. This paper introduces a set of operations tailored to support data comics narrative goals that go beyond the traditional linear, immutable storyline curated by a story author. The goals and operations include adding and removing panels into pre-defined layouts to support branching, change of perspective, or access to detail-on-demand, as well as providing and modifying data, and interacting with data representation, to support personalization and reader-defined data focus. We propose a lightweight specification language, COMICSCRIPT, for designers to add such interactivity to static comics. To assess the viability of our authoring process, we recruited six professional illustrators, designers and data comics enthusiasts and asked them to craft an interactive comic, allowing us to understand authoring workflow and potential of our approach. We present examples of interactive comics in a gallery. This initial step towards understanding the design space of interactive comics can inform the design of creation tools and experiences for interactive storytelling.
Balanced Softmax Cross-Entropy for Incremental Learning

Deep neural networks are prone to catastrophic forgetting when incrementally trained on new classes or new tasks as adaptation to the new data leads to a drastic decrease of the performance on the old classes and tasks. By using a small memory for rehearsal and knowledge distillation, recent methods have proven to be effective to mitigate catastrophic forgetting. However due to the limited size of the memory, large imbalance between the amount of data available for the old and new classes still remains which results in a deterioration of the overall accuracy of the model. To address this problem, we propose the use of the Balanced Softmax Cross-Entropy loss and show that it can be combined with exiting methods for incremental learning to improve their performances while also decreasing the computational cost of the training procedure in some cases. Experiments on the competitive ImageNet, subImageNet and CIFAR100 datasets show states-of-the-art results.
Crowd Detection and Analysis for Surveillance Videos using Deep Learning

Crowd identification and analysis has drawn a lot of attention recently, owing to a wide variety of video surveillance applications. We present a detailed review of crowd analysis and management, focusing on state-of-the-art methods for both controlled and unconstrained conditions. The paper illustrates both the advantages as well as disadvantages of state-of-the-art methods. Mass or crowd gathering can be seen at a lot of places like airports, sports stadiums, at various religious, educational, and entertainment-related events, etc. When tens of thousands of people gather in limited space, a tragedy is probably bound to happen. Automated video surveillance has become the need of the day and supports the analysis and management of data on a massive scale. It is very important to identify the presence of a crowd and detect the number of people in the gathering. This can prove very useful for the detection of sudden troupe build-up to avoid riots. Moreover, it can also be very useful in the Covid-19 pandemic situation to avoid people gathering at a place. This paper presents a system to detect the presence of a crowd by counting unique people and then performing crowd analysis. The crowd is analyzed by detecting the gender and age of people in the crowd.
Efficient Boolean Search over Encrypted Data with Reduced Leakage

Encrypted multi-maps enable outsourcing the storage of a multi-map to an untrusted server while maintaining the ability to query privately. We focus on encrypted Boolean multi-maps that support arbitrary Boolean queries over the multi-map. Kamara and Moataz [Eurocrypt’17] presented the first encrypted multi-map, BIEX, that supports CNF queries with optimal communication, worst-case sublinear search time and non-trivial leakage.
Smart Devices Based Multisensory Approach for Complex Human Activity Recognition

Sensors based Human Activity Recognition (HAR) have numerous applications in eHeath, sports, fitness assessments, ambient assisted living (AAL), human-computer interaction and many more. The human physical activity can be monitored by using wearable sensors or external devices. The usage of external devices has disadvantages in terms of cost, hardware installation, storage, computational time and lighting conditions dependencies. Therefore, most of the researchers used smart devices like smart phones, smart bands and watches which contain various sensors like accelerometer, gyroscope, GPS etc., and adequate processing capabilities. For the task of recognition, human activities can be broadly categorized as basic and complex human activities. Recognition of complex activities have received very less attention of researchers due to difficulty of problem by using either smart phones or smart watches. Other reasons include lack of sensor-based labeled dataset having several complex human daily life activities. Some of the researchers have worked on the smart phone’s inertial sensors to perform human activity recognition, whereas a few of them used both pocket and wrist positions. In this research, we have proposed a novel framework which is capable to recognize both basic and complex human activities using built-in-sensors of smart phone and smart watch. We have considered 25 physical activities, including 20 complex ones, using smart device’s built-in sensors. To the best of our knowledge, the existing literature consider only up to 15 activities of daily life.
A deceptive reviews detection model: Separated training of multi-feature learning and classification

The increasing online reviews play an essential role in the e-commerce platform, which profoundly affects the purchase decisions of consumers. However, rampant dishonest sellers manipulate other buyers or robots to post deceptive reviews for profit. Recently, the detection of deceptive reviews has attracted general research attention, which mainly comprises two directions, traditional methods based on statistics and intelligent methods based on neural networks. These methods use a single feature or multiple features for classifier design. To make full use of different features for better feature representation of detecting deceptive reviews, this paper proposes a new feature fusion strategy and verifies its performance by comparing it with other feature fusion strategies. First, we utilize three independent models for feature extraction: the TextCNN, the Bidirectional Gated Recurrent Unit (GRU), and the Self-Attention are used to learn local semantic features, temporal semantic features, and weighted semantic features of reviews, respectively. Secondly, after obtaining different feature representations from the fully connected layers of these three models, we concatenate them together to form the final documental representation. Finally, we use a full connection layer and the sigmoid function to further learn and complete deceptive review detection. Experiments on three balanced and unbalanced in-domain small datasets (hotel, restaurant, doctor) and mixed-domain datasets show that our model is superior to baselines. Experiments on large-scale data with various imbalanced proportions verify the effectiveness of our method. We also analyze the results of different datasets from the perspective of part of speech to improve the model's interpretability.