Reviewerly/exHarmony huggingface dataset:
Optimized feature space learning for generating efficient binary codes for image retrieval

In this paper, a novel approach for learning a low-dimensional optimized feature space for image retrieval with minimum intra-class variance and maximum inter-class variance is proposed. The classical approach of Linear Discriminant Analysis (LDA) is generally used for generating an optimized low-dimensional feature space for single-labeled images. Since image retrieval involves images with multiple objects, LDA cannot be directly used for dimensionality reduction and feature space optimization. This problem is addressed by utilizing the relationship between LDA and Canonical Correlation Analysis (CCA) eigenvalues to generate an optimized feature space for both single-labeled and multi-labeled images. A CCA-based network architecture which correlates the low-dimensional feature vectors with the image label vectors is proposed. We design a novel loss function such that the correlation coefficients of CCA are maximized. Our experiments prove that we could train the neural network to reach the theoretical lower bound of loss corresponding to the negative sum of the correlation coefficients. Once the optimized feature space is generated, feature vectors are binarized with the Iterative Quantization (ITQ) approach. Finally, we propose an ensemble network to generate binary codes of desired bit length for retrieval. The measurement of mean average precision shows that the proposed approach outperforms the retrieval results of other single-labeled and multi-labeled image retrieval benchmarks at same bit numbers in a considerable number of cases.
Learning Spatio-Appearance Memory Network for High-Performance Visual Tracking

Segmentation-based tracking is currently a promising tracking paradigm due to the robustness towards non-grid deformations, comparing to the traditional box-based tracking methods. However, existing segmentation-based trackers are insufficient in modeling and exploiting dense pixel-wise correspondence across frames. To overcome these limitations, this paper presents a novel segmentation-based tracking architecture equipped with spatio-appearance memory networks. The appearance memory network utilizes spatio-temporal non-local similarity to propagate segmentation mask to the current frame, which can effectively capture long-range appearance variations and we further treat discriminative correlation filter as spatial memory bank to store the mapping between feature map and spatial map. Moreover, mutual promotion on dual memory networks greatly boost the overall tracking performance. We further propose a dynamic memory machine (DMM) which employs the Earth Mover’s Distance (EMD) to reweight memory samples. Without bells and whistles, our simple-yet-effective tracking architecture sets a new state-of-the-art on six tracking benchmarks. Besides, our approach achieves comparable results on two video object segmentation benchmarks. Code and model are released at https://github.com/phiphiphi31/DMB.
Large-Scale Crowdsourcing Subjective Quality Evaluation of Learning-Based Image Coding

Learning-based image codecs produce different compression artifacts, when compared to the blocking and blurring degradation introduced by conventional image codecs, such as JPEG, JPEG 2000 and HEIC. In this paper, a crowdsourcing based subjective quality evaluation procedure was used to benchmark a representative set of end-to-end deep learning-based image codecs submitted to the MMSP'2020 Grand Challenge on Learning-Based Image Coding and the JPEG AI Call for Evidence. For the first time, a double stimulus methodology with a continuous quality scale was applied to evaluate this type of image codecs. The subjective experiment is one of the largest ever reported including more than 240 pair-comparisons evaluated by 118 naïve subjects. The results of the benchmarking of learning-based image coding solutions against conventional codecs are organized in a dataset of differential mean opinion scores along with the stimuli and made publicly available.
Ant Cat Swarm Optimization-Enabled Deep Recurrent Neural Network for Big Data Classification Based on Map Reduce Framework

Abstract The amount of data generated is increasing day by day due to the development in remote sensors, and thus it needs concern to increase the accuracy in the classification of the big data. Many classification methods are in practice; however, they limit due to many reasons like its nature for data loss, time complexity, efficiency and accuracy. This paper proposes an effective and optimal data classification approach using the proposed Ant Cat Swarm Optimization-enabled Deep Recurrent Neural Network (ACSO-enabled Deep RNN) by Map Reduce framework, which is the incorporation of Ant Lion Optimization approach and the Cat Swarm Optimization technique. To process feature selection and big data classification, Map Reduce framework is used. The feature selection is performed using Pearson correlation-based Black hole entropy fuzzy clustering. The classification in reducer part is performed using Deep RNN that is trained using a developed ACSO scheme. It classifies the big data based on the reduced dimension features to produce a satisfactory result. The proposed ACSO-based Deep RNN showed improved results with maximal specificity of 0.884, highest accuracy of 0.893, maximal sensitivity of 0.900 and the maximum threat score of 0.827 based on the Cleveland dataset.
Green Universities in an Orange Economy: New Campus Policy

Changes in modern economic spaces of different countries of the world, global challenges stimulate interest in the goals in the field of sustainable development of territories. Elements of the theory of innovative industries are used to solve problems in classical segments of the economy associated with ensuring their growth and development. This theory is built on concept of orange economy where creative and intellectual potential are the main resources. The purpose of the article is to analyze the experience of UrFU in solving problems in the field of sustainable development and developing the concept of a “green university” and “green campus”, which imply an improvement in the quality of the environment. Methods of content analysis, qualitative and quantitative research methods were used in the work such as methods of systematization, analysis of documents and statistical information, foresight session. We have identified the main goals of the university campus policy; identified and substantiated the main categories that meet the challenges of the future orange economy. The authors analyzed the results of a foresight session with experts in the field of sustainable development and creative economics, which made it possible to develop forecasts for the development of the university with a planning horizon till 2030 for the implementation of a new campus policy aimed at creating a green university. The article discusses a predictive model for creating a university campus on the example of the Ural Federal University with possible development as part of the orange economy concept.
Multiple Leader PBFT Based Blockchain Architecture for IoT Domains: Invited Paper

In this paper we propose to implements blockchain technology for Internet of Things (IoT) networks that use Practical Byzantine Fault Tolerance (PBFT) consensus algorithm. To eliminate the reliance on a single leader and improve performance, we propose to allow multiple leaders to propose request batches independently and, possibly, concurrently, by using different overlay networks. Nodes participate in parallel consensus rounds without contention, and they only contend with others to reserve the next available spot for the atomic insertion of a new transaction batch or block into the replicated blockchain ledger. We develop an analytical model for the multiple leader PBFT ordering service by using a Discrete-time Markov chain. Our evaluations show that our model outperforms the original multiple entry point PBFT protocol in a wide range of parameter values, and that it scales well with the number of orderer nodes in the PBFT committee and block arrival rate.
Generating Pseudo Connectives with MLMs for Implicit Discourse Relation Recognition

Due to the lack of connectives, the recognition of implicit discourse relations faces a big challenge. An early attempt overcomes this difficulty by predicting connectives with the use of the statistical language model. Recent years have witnessed the great success of masked language models (MLM). Then a new problem naturally arises, i.e., how can connectives benefit implicit discourse relation classification from such models? In this paper, we address this problem by developing a novel framework to generate the pseudo connectives using the pre-trained MLM. The key idea is to treat the absent connectives as missing words between two arguments and produce the pseudo connective from its contexts by fine-tuning MLM on the classification task. Moreover, we leverage the real connectives in explicit discourse relations to supervise the generation of pseudo connectives. Extensive experiments show that our model achieves the state-of-the-art performance on the PDTB benchmark.
Simple Pair Pose - Pairwise Human Pose Estimation in Dense Urban Traffic Scenes

Despite the success of deep learning, human pose estimation remains a challenging problem in particular in dense urban traffic scenarios. Its robustness is important for followup tasks like trajectory prediction and gesture recognition. We are interested in human pose estimation in crowded scenes with overlapping pedestrians, in particular pairwise constellations. We propose a new top-down method that relies on pairwise detections as input and jointly estimates the two poses of such pairs in a single forward pass within a deep convolutional neural network. As availability of automotive datasets providing poses and a fair amount of crowded scenes is limited, we extend the EuroCity Persons dataset by additional images and pose annotations. With 46,975 images and poses of 279,329 persons our new EuroCity Persons Dense Pose dataset is the largest pose dataset recorded from a moving vehicle. In our experiments using this dataset we show improved performance for poses of pedestrian pairs in comparison with a state of the art method for human pose estimation in crowds.
A Simulation-Based Benchmark for Behavioral Anomaly Detection in Autonomous Vehicles

A central challenge in the development of autonomous vehicles (AVs) is identifying and responding to anomalous scenarios. Due to the complexity of urban streets, environmental conditions, and the behavior of other actors, AVs encounter out-of-distribution events at a non-trivial rate (often termed the long-tail problem). Anomalous situations may be statically perceivable (e.g. an overturned truck, or a mounted policeman) or require nuanced behavioral inference (e.g. vehicles swerving to avoid broken glass on the roadway). Despite the importance of robustly identifying these situations, there are few resources for developing or benchmarking new methods in the research community. Here we introduce a framework for generating anomalous and nominal scenarios using a domain-specific scenario description language (Scenic) and a driving simulation environment (CARLA) from multiple viewpoints. We introduce four classes of roadway anomalies and establish a benchmark and baselines for the development of new methods. Crucially, our benchmark supports the development of both sensor-based and semantic approaches. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> The code and related documentation for this work is available at https://github.com/t27/carla-scenic-data-collector
Frequency-hiding order-preserving encryption with small client storage

The range query on encrypted databases is usually implemented using the order-preserving encryption (OPE) technique which preserves the order of plaintexts. Since the frequency leakage of plaintexts makes OPE vulnerable to frequency-analyzing attacks, some frequency-hiding order-preserving encryption (FH-OPE) schemes are proposed. However, existing FH-OPE schemes require either the large client storage of size O ( n ) or O (log n ) rounds of interactions for each query, where n is the total number of plaintexts. To this end, we propose a FH-OPE scheme that achieves the small client storage without additional client-server interactions. In detail, our scheme achieves O ( N ) client storage and 1 interaction per query, where N is the number of distinct plaintexts and N ≤ n . Especially, our scheme has a remarkable performance when N ≪ n . Moreover, we design a new coding tree for producing the order-preserving encoding which indicates the order of each ciphertext in the database. The coding strategy of our coding tree ensures that encodings update in the low frequency when inserting new ciphertexts. Experimental results show that the single round interaction and low-frequency encoding updates make our scheme more efficient than previous FH-OPE schemes.
SportsSum2.0: Generating High-Quality Sports News from Live Text Commentary

Sports game summarization aims to generate news articles from live text commentaries. A recent state-of-the-art work, SportsSum, not only constructs a large benchmark dataset, but also proposes a two-step framework. Despite its great contributions, the work has three main drawbacks: 1) the noise existed in SportsSum dataset degrades the summarization performance; 2) the neglect of lexical overlap between news and commentaries results in low-quality pseudo-labeling algorithm; 3) the usage of directly concatenating rewritten sentences to form news limits its practicability. In this paper, we publish a new benchmark dataset SportsSum2.0, together with a modified summarization framework. In particular, to obtain a clean dataset, we employ crowd workers to manually clean the original dataset. Moreover, the degree of lexical overlap is incorporated into the generation of pseudo labels. Further, we introduce a reranker-enhanced summarizer to take into account the fluency and expressiveness of the summarized news. Extensive experiments show that our model outperforms the state-of-the-art baseline.
Success Factors when Transitioning to Continuous Deployment in Software-Intensive Embedded Systems

Continuous Deployment is the practice to deploy software more frequently to customers and learn from their usage. The aim is to introduce new functionality and features in an additive way to customers as soon as possible. While Continuous Deployment is becoming popular among web and cloud-based software development organizations, the adoption of continuous deployment within the software-intensive embedded systems industry is still limited.In this paper, we conducted a case study at a multinational telecommunications company focusing on the Third Generation Radio Access Network (3G RAN) embedded software. The organization has transitioned to Continuous Deployment where the software’s deployment cycle has been reduced to 4 weeks from 24 weeks. The objective of this paper is to identify what does success means when transitioning to continuous deployment and the success factors that companies need to attend to when transitioning to continuous deployment in a large-scale embedded software.
Personal Identifiability and Obfuscation of User Tracking Data From VR Training Sessions

Recent research indicates that user tracking data from virtual reality (VR) experiences can be used to personally identify users with degrees of accuracy as high as 95%. However, these results indicating that VR tracking data should be understood as personally identifying data were based on observing 360° videos. In this paper, we present results based on sessions of user tracking data from an ecologically valid VR training application, which indicate that the prior claims may not be as applicable for identifying users beyond the context of observing 360° videos. Our results indicate that the degree of identification accuracy notably decreases between VR sessions. Furthermore, we present results indicating that user tracking data can be obfuscated by encoding positional data as velocity data, which has been successfully used to predict other user experience outcomes like simulator sickness and knowledge acquisition. These results, which show identification accuracies were reduced by more than half, indicate that velocity-based encoding can be used to reduce identifiability and help protect personal identifying data.
Detecting Adversarial Patch Attacks through Global-local Consistency

Recent works have well-demonstrated the threat of adversarial patch attacks to real-world vision media systems. By arbitrarily modifying pixels within a small restricted area in the image, adversarial patches can mislead neural-network-based image classifiers. In this paper, we propose a simple but very effective approach to detect adversarial patches based on an interesting observation called global-local consistency. We verify this insight and propose to use Random-Local-Ensemble (RLE) strategy to further enhance it in the detection. The proposed method is trivial to implement and can be applied to protect any image classification models. Experiments on two popular datasets show that our algorithm can accurately detect the adversarial patches while maintaining high clean accuracy. Moreover, unlike the prior detection approaches which can be easily broken by adaptive attacks, our method is proved to have high robustness when facing adaptive attacks.
Fusion of Evidential CNN Classifiers for Image Classification

We propose an information-fusion approach based on belief functions to combine convolutional neural networks. In this approach, several pre-trained DS-based CNN architectures extract features from input images and convert them into mass functions on different frames of discernment. A fusion module then aggregates these mass functions using Dempster's rule. An end-to-end learning procedure allows us to fine-tune the overall architecture using a learning set with soft labels, which further improves the classification performance. The effectiveness of this approach is demonstrated experimentally using three benchmark databases.
DiagViB-6: A Diagnostic Benchmark Suite for Vision Models in the Presence of Shortcut and Generalization Opportunities

Common deep neural networks (DNNs) for image classification have been shown to rely on shortcut opportunities (SO) in the form of predictive and easy-to-represent visual factors. This is known as shortcut learning and leads to impaired generalization. In this work, we show that common DNNs also suffer from shortcut learning when predicting only basic visual object factors of variation (FoV) such as shape, color, or texture. We argue that besides shortcut opportunities, generalization opportunities (GO) are also an inherent part of real-world vision data and arise from partial independence between predicted classes and FoVs. We also argue that it is necessary for DNNs to exploit GO to overcome shortcut learning. Our core contribution is to introduce the Diagnostic Vision Benchmark suite DiagViB-6, which includes datasets and metrics to study a network’s shortcut vulnerability and generalization capability for six independent FoV. In particular, DiagViB-6 allows controlling the type and degree of SO and GO in a dataset. We benchmark a wide range of popular vision architectures and show that they can exploit GO only to a limited extent.
MLRMV: Multi-layer representation for multi-view action recognition

Daily action recognition has gained much interest in computer vision. However, viewpoint changes will lead to sizable intra-class differences in the same action. To deal with this problem, we propose a novel multi-view daily action recognition approach based on the multi-layer representation. In use of motion atoms and motion phrases, we construct the middle-level feature representations in multi-view daily actions. A multi-view unsupervised discriminative clustering method is proposed for constructing motion atoms, and the classification accuracy of motion atoms is improved by jointly learning atom dictionaries and the classifier. Moreover, we present discontinuous temporal scale motion phrases and a grading mechanism of motion phrases to strengthen the representative ability of motion phrases and the final recognition accuracy. Finally, the experimental results based on the WVU dataset, the NTU RGB-D dataset, and N-UCLA dataset show that the proposed methods have the state-of-the-art performance, compared with the classic methods such as IDT, MoFAP, JLMF, and so on.
Real-time Human Activity Recognition Using ResNet and 3D Convolutional Neural Networks

In computer vision-based applications, the recognition of human activity is always a standard problem. Nowadays, activity recognition is more possible and accurate due to good development in artificial neural networks like convolutional neural network CNN. In many recent works, the recognition model architecture use CNN and long short-term memory units (LSTM) - attention models to extract spatial and temporal features from the input video. This particular work is related to real-time human activity recognition by Resnet and 3D CNN without the involvement of the LSTM- attention model. Here the 2D Resnet is modified to 3D CNN to achieve better human activity recognition accuracy. The wide range of data information from the kinetics dataset can avoid overfitting issues during the training period. And the combination of Resnet and 3D CNN can enhance the accuracy of recognition. As a consequence, a method for detecting, monitoring, and recognizing real-time human motion has been developed.
Small object detection combining attention mechanism and a novel FPN

Since small objects occupy less pixels in the image and are difficult to recognize. Small object detection has always been a research difficulty in the field of computer vision. Aiming at the problems of low sensitivity and poor detection performance of YOLOv3 for small objects. AFYOLO, which is more sensitive to small objects detection was proposed in this paper. Firstly, the DenseNet module is introduced into the low-level layers of backbone to enhance the transmission ability of objects information. At the same time, a new mechanism combining channel attention and spatial attention is introduced to improve the feature extraction ability of the backbone. Secondly, a new feature pyramid network (FPN) is proposed to better obtain the features of small objects. Finally, ablation studies on ImageNet classification task and MS-COCO object detection task verify the effectiveness of the proposed attention module and FPN. The results on Wider Face datasets show that the AP of the proposed method is 11.89%higher than that of YOLOv3 and 8.59%higher than that of YOLOv4. All of results show that AFYOLO has better ability for small object detection.