Reviewerly/exHarmony huggingface dataset:
A regional distance regression network for monocular object distance estimation

Monocular pipelines are convenient and cost-effective solutions for object distance estimation in 3D vision. Current methods for monocular object distance estimation either perform inaccurately or require heavy work on data collection. In this paper, we propose a network with R-CNN based structure to implement object detection and distance estimation simultaneously. We append an efficient branch to integrate the information of camera extrinsic parameters with RGB data in our network. Further, optimized multi-scale feature is utilized to enrich the representation power of deep feature, hence to enhance the estimation accuracy. Finally, several regression methods are explored to improve distance estimation results. We train and validate our network on KITTI object dataset, and compare with other methods to show that our method is accurate and easy to train. To prove the generality of our method under other scenarios, we construct a dataset of surveillance scenes, and conduct similar experiments on this dataset.
A new efficient permutation-diffusion encryption algorithm based on a chaotic map

Nowadays, the internet is the main channel for private communication. The use of a public channel for private communications increases the importance of cryptography to protect the information. The most commonly shared archives are images, for which traditional algorithms are not suitable due to the file’s large size and algorithms’ high computational cost. In this scenario, chaos theory surges as a promising alternative for image encryption due to intrinsic features of chaos, leading to the development of many chaotic ciphers for image encryption. However, many of them are either insecure or ineffective, motivating us to present a new chaotic cipher that satisfies both of these requirements. To ensure peak performance, a comparison of chaotic maps is made to select the one most conductible to a high throughout, while architecture measures such as the use of the permutation-diffusion architecture and a hash function are taken to ensure security. Extensive tests applied show that the cipher is secure against various forms of attacks such as statistical, differential, noise, occlusion and chosen-plaintext, while the throughput analysis shows that our cipher compares favorably to other recently presented chaotic ciphers in literature.
Empirical Study: Visual Analytics for Comparing Stacking to Blending Ensemble Learning

Stacked generalization (also called stacking) is an ensemble method in machine learning that uses a metamodel to combine the predictive results of heterogeneous base models arranged in at least one layer. K-fold cross-validation is employed at the various stages of training in this method. Nonetheless, another validation strategy is to try out several splits of data leading to different train and test sets for the base models and then use only the latter to train the metamodel—this is known as blending. In this work, we present a modification of an existing visual analytics system, entitled StackGenVis, that now supports the process of composing robust and diverse ensembles of models with both aforementioned methods. We have built multiple ensembles using our system with the two respective methods, and we tested the performance with six small- to large-sized data sets. The results indicate that stacking is significantly more powerful than blending based on three performance metrics. However, the training times of the base models and the final ensembles are lower and more stable during various train/test splits in blending rather than stacking.
From Raw Text to Enhanced Universal Dependencies: The Parsing Shared Task at IWPT 2021

We describe the second IWPT task on end-toend parsing from raw text to Enhanced Universal Dependencies.We provide details about the evaluation metrics and the datasets used for training and evaluation.We compare the approaches taken by participating teams and discuss the results of the shared task, also in comparison with the first edition of this task.
Forecasting Video QoE With Deep Learning From Multivariate Time-Series

The end users’ satisfactory Quality of Experience (QoE) is a fundamental criterion for networked video service providers such as video-on-demand providers (Netflix, YouTube, etc.), cloud gaming providers (Google Stadia, PlayStation Now, etc.) and videoconferencing providers (Zoom, Microsoft Teams, etc.). To know the QoE, providers today typically predict it from the Quality of Service (QoS) parameters or the client-side's actual QoE metrics measured at the current time-step. But the former does not precisely reflect the users' experience, and the latter has a delay between QoE measurements at the client-side and the user's current experience. Mitigating this delay can provide a noticeable improvement in the delivery system's performance. For example, accurate forecasting of QoE for the near future allows the service management system to take a proactive approach and fix delivery issues before they become a noticeable problem at the end user, or at least reduce overall QoE degradation. QoE forecasting can also be used in rate adaptation in DASH or resource allocation in wireless networks. In this paper, we propose a method to prognosticate QoE metrics. Using data collected from an industry video streaming testbed for three different classes, we define a multivariate time series forecasting problem. We then model a hybrid state-of-the-art deep learning method, BiLSTM-CNN, to forecast the QoE metrics in advance. Evaluation of our proposed method compared to four other well-known ML models of Support Vector Regression (SVR), Multi-Layer Perceptron (MLP), Long short-term memory (LSTM), and Bidirectional LSTM (BiLSTM) demonstrates the superior performance of our proposed method.
Building Knowledge Capacity for Quantum Computing in Engineering Education

The Quantum industry is currently at an embryonic stage. If it is to grow, it will require new markets, and, a workforce with the requisite skills and knowledge to support it. Anticipating this potential growth, this paper will explore capacity building within engineering education on the subject of Quantum Computing (QC). This work has two aims. On the one hand, it seeks to illustrate the need for developing education on the subject, inferred by trends in open literature. On the other hand, it seeks to suggest a starting point for quantum computing education in higher education. Since 2018, a sharp incline can be observed in the number of publications on topics related to QC. These publications are arising within several fields related to engineering including, but not limited to, material science, chemistry and computer science. In response to this trend, this paper will evaluate several third party educational approaches to teaching emergent technologies with a view to developing a model for teaching QC. Due to a lack of precedent in a wide range of industry applications and the current limitations in the state-of-the-art of this technology, the educational model proposed will be one that exploits imagination, as opposed to knowledge acquisition, in the pursuit of new knowledge building.
EQFace: A Simple Explicit Quality Network for Face Recognition

As the deep learning makes big progresses in still-image face recognition, unconstrained video face recognition is still a challenging task due to low quality face images caused by pose, blur, occlusion, illumination etc. In this paper we propose a network for face recognition which gives an explicit and quantitative quality score at the same time when a feature vector is extracted. To our knowledge this is the first network that implements these two functions in one network online. This network is very simple by adding a quality network branch to the baseline network of face recognition. It does not require training datasets with annotated face quality labels. We evaluate this network on both still-image face datasets and video face datasets and achieve the state-of-the-art performance in many cases. This network enables a lot of applications where an explicit face quality scpre is used. We demonstrate three applications of the explicit face quality, one of which is a progressive feature aggregation scheme in online video face recognition. We design an experiment to prove the benefits of using the face quality in this application. Code will be available at https://github.com/deepcam-cn/facequality.
FlipReID: Closing the Gap Between Training and Inference in Person Re-Identification

Since neural networks are data-hungry, incorporating data augmentation in training is a widely adopted technique that enlarges datasets and improves generalization. On the other hand, aggregating predictions of multiple augmented samples (i.e., test-time augmentation) could boost performance even further. In the context of person re-identification models, it is common practice to extract embeddings for both the original images and their horizontally flipped variants. The final representation is the mean of the aforementioned feature vectors. However, such scheme results in a gap between training and inference, i.e., the mean feature vectors calculated in inference are not part of the training pipeline. In this study, we devise the FlipReID structure with the flipping loss to address this issue. More specifically, models using the FlipReID structure are trained on the original images and the flipped images simultaneously, and incorporating the flipping loss minimizes the mean squared error between feature vectors of corresponding image pairs. Extensive experiments show that our method brings consistent improvements. In particular, we set a new record for MSMT17 which is the largest person re-identification dataset. The source code is available at https://github.com/nixingyang/FlipReID.
Violence detection using pre-trained models

Violence detection aims at recognizing whether a violent action has happened. This field gained widespread popularity since there is a need to find applicable and automatic violence detection methods which explored visual data received from surveillance cameras installed in different areas. In this paper, we employed pre-trained deep neural networks in order to present a low-complexity method for violence detection. The extracted features from pre-trained models have been pooled and fed into a fully connected network in order to detect whether a violent action has occurred. As pre-trained models, the results of both ResNet-50 and VGG16 are explored in the proposed approach. We evaluate the effectiveness of the method on four public datasets. The experimental results depict the efficiency of the low-complexity proposed approach in comparison with other approaches using time-consuming networks like recurrent ones.
Visual question answering in the medical domain based on deep learning approaches: A comprehensive study

Visual Question Answering (VQA) in the medical domain has attracted more attention from research communities in the last few years due to its various applications. This paper investigates several deep learning approaches in building a medical VQA system based on ImageCLEF's VQA-Med dataset, which consists of about 4K images with about 15K question-answer pairs. Due to the wide variety of the images and questions included in this dataset, the proposed model is a hierarchical one consisting of many sub-models, each tailored to handle certain questions. For that, a special model is built to classify the questions into four categories, where each category is handled by a separate sub-model. At their core, all of these models consist of pre-trained Convolution Neural Networks (CNN). In order to get the best results, extensive experiments are performed and various techniques are employed including Data Augmentation (DA), Multi-Task Learning (MTL), Global Average Pooling (GAP), Ensembling, and Sequence to Sequence (Seq2Seq) models. Overall, the final model achieves 60.8 accuracy and 63.4 BLEU score, which are competitive with the state-of-the-art results despite using less demanding and simpler sub-models.
Encrypted Value Iteration and Temporal Difference Learning over Leveled Homomorphic Encryption

We consider an architecture of confidential cloud-based control synthesis based on Homomorphic Encryption (HE). Our study is motivated by the recent surge of data-driven control such as deep reinforcement learning, whose heavy computational requirements often necessitate an outsourcing to the third party server. To achieve more flexibility than Partially Homomorphic Encryption (PHE) and less computational overhead than Fully Homomorphic Encryption (FHE), we consider a Reinforcement Learning (RL) architecture over Leveled Homomorphic Encryption (LHE). We first show that the impact of the encryption noise under the Cheon-Kim-Kim-Song (CKKS) encryption scheme on the convergence of the model-based tabular Value Iteration (VI) can be analytically bounded. We also consider secure implementations of TD(0), SARSA(0) and Z-learning algorithms over the CKKS scheme, where we numerically demonstrate that the effects of the encryption noise on these algorithms are also minimal.
Domain-matched Pre-training Tasks for Dense Retrieval

Barlas Oguz, Kushal Lakhotia, Anchit Gupta, Patrick Lewis, Vladimir Karpukhin, Aleksandra Piktus, Xilun Chen, Sebastian Riedel, Scott Yih, Sonal Gupta, Yashar Mehdad. Findings of the Association for Computational Linguistics: NAACL 2022. 2022.
Investigating Shift Variance of Convolutional Neural Networks in Ultrasound Image Segmentation

While accuracy is an evident criterion for ultrasound image segmentation, output consistency across different tests is equally crucial for tracking changes in regions of interest in applications such as monitoring the patients' response to treatment, measuring the progression or regression of the disease, reaching a diagnosis, or treatment planning. Convolutional neural networks (CNNs) have attracted rapidly growing interest in automatic ultrasound image segmentation recently. However, CNNs are not shift-equivariant, meaning that, if the input translates, e.g., in the lateral direction by one pixel, the output segmentation may drastically change. To the best of our knowledge, this problem has not been studied in ultrasound image segmentation or even more broadly in ultrasound images. Herein, we investigate and quantify the shift-variance problem of CNNs in this application and further evaluate the performance of a recently published technique, called BlurPooling, for addressing the problem. In addition, we propose the Pyramidal BlurPooling method that outperforms BlurPooling in both output consistency and segmentation accuracy. Finally, we demonstrate that data augmentation is not a replacement for the proposed method. Source code is available at http://code.sonography.ai.
Motion estimation in hazy videos

Motion estimation is the basic need for the success of many video analysis algorithms such as moving object detection, human activity recognition, etc. Most of the motion estimation algorithms are prone to weather conditions and thus, they fail to estimate the motion in degraded weather. Severe weather situations like snow, rain, haze, smog, etc., degrades the performance and reliability of video analysis algorithms. In this paper, we have analyzed the effect of the haze on motion estimation in hazy videos. We propose a cascaded architecture i.e. haze removal followed by optical flow for motion estimation in hazy videos. The proposed image de-hazing network is build upon the Residual and Inception module concepts and named as ResINet. Further, an optical flow is utilized to estimate the motion information. We have carried out the visual analysis to validate the proposed approach for motion estimation in hazy videos. Also, to validate the proposed ResINet for de-hazing, we carried out the quantitative analysis on two benchmark image de-hazing datasets.
Designing an extended smart classroom: An approach to game‐based learning for IoT

This article presents a game-based learning model implemented in a smart learning environment. The goal is to determine students' interest, willingness to participate, and impressions about game-based learning, which takes place in interaction with the smart classroom. The educational game was designed and implemented by harnessing advanced web, mobile, Internet of Things (IoTs) technologies, and augmented reality and integrated into an e-learning ecosystem based on Moodle platform. The game aimed to test students' knowledge in the field of IoT. For the evaluation, an experiment was conducted within the IoT-related courses at the Faculty of Organizational Sciences, University of Belgrade. The results show that the proposed approach has a positive impact on the learning process and that the integration of educational games into common learning management systems is a good practice for extending common formal learning models. The proposed approach is expected to contribute to introducing smart learning environments into the learning process in higher education.
3D Scene Compression through Entropy Penalized Neural Representation Functions

Some forms of novel visual media enable the viewer to explore a 3D scene from essentially arbitrary viewpoints, by interpolating between a discrete set of original views. Compared to 2D imagery, these types of applications require much larger amounts of storage space, which we seek to reduce. Existing approaches for compressing 3D scenes are often based on a separation of compression and rendering: each of the original views is compressed using traditional 2D image formats; the receiver decompresses the views and then performs the rendering. We unify these steps by directly compressing an implicit representation of the scene, a function that maps spatial coordinates to a radiance vector field, which can then be queried to render arbitrary viewpoints. The function is implemented as a neural network and jointly trained for reconstruction as well as compressibility, in an end-to-end manner, with the use of an entropy penalty on the parameters. Our method significantly outperforms a state-of-the-art conventional approach for scene compression, achieving simultaneously higher quality reconstructions and lower bitrates. Furthermore, we show that the performance at lower bitrates can be improved by jointly representing multiple scenes using a soft form of parameter sharing.
Privacy Preserving Face Recognition in Cloud Robotics: A Comparative Study

Real-time robotic applications encounter the robot on board resources’ limitations. The speed of robot face recognition can be improved by incorporating cloud technology. However, the transmission of data to the cloud servers exposes the data to security and privacy attacks. Therefore, encryption algorithms need to be set up. This paper aims to study the security and performance of potential encryption algorithms and their impact on the deep-learning-based face recognition task’s accuracy. To this end, experiments are conducted for robot face recognition through various deep learning algorithms after encrypting the images of the ORL database using cryptography and image-processing based algorithms.
Fast, Nonlocal and Neural: A Lightweight High Quality Solution to Image Denoising

With the widespread application of convolutional neural networks (CNNs), the traditional model based denoising algorithms are now outperformed. However, CNNs face two problems. First, they are computationally demanding, which makes their deployment especially difficult for mobile terminals. Second, experimental evidence shows that CNNs often over-smooth regular textures present in images, in contrast to traditional non-local models. In this letter, we propose a solution to both issues by combining a nonlocal algorithm with a lightweight residual CNN. This solution gives full latitude to the advantages of both models. We apply this framework to two GPU implementations of classic nonlocal algorithms (NLM and BM3D) and observe a substantial gain in both cases, performing better than the state-of-the-art with low computational requirements. Our solution is between 10 and 20 times faster than CNNs with equivalent performance and attains higher PSNR. In addition the final method shows a notable gain on images containing complex textures like the ones of the MIT Moire dataset.
Label Disentangled Analysis for unsupervised visual domain adaptation

Labeling a large number of samples in machine learning is often a prohibitively time-consuming task. Unsupervised domain adaptation (UDA) aims to learn a model to classify or represent the target domain by borrowing sufficiently labeled data from semantic adjacent but differently distributed source domains. However, existing DA models ignore an essential problem, i.e. the label dominated supervision signal easily makes the learning model/network tend to memorize the data, but not to learn essential image features, which makes DA unexplainable on what makes transfer a success. With this motivation, we have a core idea to disentangle/erase the label information from image features for easier DA task. To that end, this paper for the first time proposes the "label-erased" and "label-reconstructed" feature distributions by establishing their bidirectional mapping, and models them respectively for domain adaptation with the philosophy of divide and conquer. Specifically, in this paper, we propose a Label Disentangled Analysis (LDA) approach, consisting of three novel parts: (1) disentangling (decoupling) the labels for label-erased (label-irrelevant) features from domain data, (2) aligning the label-erased image features and label-reconstructed features between domains, and (3) cross-associating the aligned image and label features together to ensure the class discrimination of data. The above process can be summarized as three actions: label disentanglement (decoupling), feature-label joint adaptation (alignment), and feature-label cross association (cross-coupling). Extensive experiments verify that the proposed LDA outperforms the SOTAs on a number of benchmarks and is also comparable to several advanced deep transfer methods.
Predicting combat outcomes and optimizing armies in StarCraft II by deep learning

Real-time strategy (RTS) games’ nature that, more complex than the turn-based, tabletop games such as Go, has been spotlighted in the field of artificial intelligence (AI) due to its similarity with real-world problems. In StarCraft II , agents cannot make decisions and control until they evaluate and compare the expected outcome of a choice. Among the ways to evaluate outcomes, combat models are one of the active areas of research to this problem, which is a basis for decision-making. The battlefield of combat needs to be considered in combat models because they have enough influence to overturn the outcome of the battle. However, its effect has not been sufficiently examined. We introduce a combat winner predictor that utilizes battlefield and troop information. Furthermore, we propose a constrained optimization framework with gradient updates to optimize unit-combinations based on the combat winner predictor. Experiments demonstrate the robustness and rapidness of the proposed methods in large-scale combat datasets on various battlefields of StarCraft II . The proposed framework achieved better accuracy in prediction and retrieved winning unit-combinations faster. Incorporating these frameworks into AI agents can improve the AI’s decision-making power. • Decision-making is a key-problem in real-time strategy games. • Proposed method enhanced StarCraft II AI’s decision-making ability regardless of battlefields. • A neural-network-based surrogate model resulted in rapid decision-making.