Reviewerly/exHarmony huggingface dataset:
Collective Entity Alignment via Adaptive Features

Entity alignment (EA) identifies entities that refer to the same real-world object but locate in different knowledge graphs (KGs), and has been harnessed for KG construction and integration. When generating EA results, current solutions treat entities independently and fail to take into account the interdependence between entities. To fill this gap, we propose a collective EA framework. We first employ three representative features, i.e., structural, semantic and string signals, which are adapted to capture different aspects of the similarity between entities in heterogeneous KGs. In order to make collective EA decisions, we formulate EA as the classical stable matching problem, which is further effectively solved by deferred acceptance algorithm. Our proposal is evaluated on both cross-lingual and mono-lingual EA benchmarks against state-of-the-art solutions, and the empirical results verify its effectiveness and superiority.
Using mobile devices to enhance inquiry-based learning processes

Technological advances offer the possibility of using mobile devices to enrich learning environments with multimedia content. Although physical experiments play a key role in science learning, little is known about integrating multimedia learning in physical experimental processes, especially in realistic classroom settings. In our approach, students use tablets to investigate motion with an application providing multiple representations of the measurement data. We present the results of a cluster-randomized controlled study with high school students (average age = 15.6, N = 286) conducted between 2017 and 2018 in 11 secondary schools in Germany. Conceptual understanding of physics, cognitive load, and assessment of teacher behavior were surveyed. Multilevel regression analysis revealed that this approach leads to a significant reduction of extraneous cognitive load and to greater conceptual knowledge. A confirmatory path analysis revealed that these effects can be traced back to the treatment and are not significantly influenced by teacher behavior.
Ethainter: a smart contract security analyzer for composite vulnerabilities

Smart contracts on permissionless blockchains are exposed to inherent security risks due to interactions with untrusted entities. Static analyzers are essential for identifying security risks and avoiding millions of dollars worth of damage.
TCTS: A Task-Consistent Two-Stage Framework for Person Search

The state of the art person search methods separate person search into detection and re-ID stages, but ignore the consistency between these two stages. The general person detector has no special attention on the query target; The re-ID model is trained on hand-drawn bounding boxes which are not available in person search. To address the consistency problem, we introduce a Task-Consist Two-Stage (TCTS) person search framework, includes an identity-guided query (IDGQ) detector and a Detection Results Adapted (DRA) re-ID model. In the detection stage, the IDGQ detector learns an auxiliary identity branch to compute query similarity scores for proposals. With consideration of the query similarity scores and foreground score, IDGQ produces query-like bounding boxes for the re-ID stage. In the re-ID stage, we predict identity labels of detected bounding boxes, and use these examples to construct a more practical mixed train set for the DRA model. Training on the mixed train set improves the robustness of the re-ID stage to inaccurate detection. We evaluate our method on two benchmark datasets, CUHK-SYSU and PRW. Our framework achieves 93.9% of mAP and 95.1% of rank1 accuracy on CUHK-SYSU, outperforming the previous state of the art methods.
Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks

Batch Normalization (BN) uses mini-batch statistics to normalize the activations during training, introducing dependence between mini-batch elements. This dependency can hurt the performance if the mini-batch size is too small, or if the elements are correlated. Several alternatives, such as Batch Renormalization and Group Normalization (GN), have been proposed to address this issue. However, they either do not match the performance of BN for large batches, or still exhibit degradation in performance for smaller batches, or introduce artificial constraints on the model architecture. In this paper we propose the Filter Response Normalization (FRN) layer, a novel combination of a normalization and an activation function, that can be used as a replacement for other normalizations and activations. Our method operates on each activation channel of each batch element independently, eliminating the dependency on other batch elements. Our method outperforms BN and other alternatives in a variety of settings for all batch sizes. FRN layer performs 0.7-1.0% better than BN on top-1 validation accuracy with large mini-batch sizes for Imagenet classification using InceptionV3 and ResnetV2-50 architectures. Further, it performs >1% better than GN on the same problem in the small mini-batch size regime. For object detection problem on COCO dataset, FRN layer outperforms all other methods by at least 0.3-0.5% in all batch size regimes.
Multi-modal Dense Video Captioning

Dense video captioning is a task of localizing interesting events from an untrimmed video and producing textual description (captions) for each localized event. Most of the previous works in dense video captioning are solely based on visual information and completely ignore the audio track. However, audio, and speech, in particular, are vital cues for a human observer in understanding an environment. In this paper, we present a new dense video captioning approach that is able to utilize any number of modalities for event description. Specifically, we show how audio and speech modalities may improve a dense video captioning model. We apply automatic speech recognition (ASR) system to obtain a temporally aligned textual description of the speech (similar to subtitles) and treat it as a separate input alongside video frames and the corresponding audio track. We formulate the captioning task as a machine translation problem and utilize recently proposed Transformer architecture to convert multi-modal input data into textual descriptions. We demonstrate the performance of our model on ActivityNet Captions dataset. The ablation studies indicate a considerable contribution from audio and speech components suggesting that these modalities contain substantial complementary information to video frames. Furthermore, we provide an in-depth analysis of the ActivityNet Caption results by leveraging the category tags obtained from original YouTube videos. Code is publicly available: github.com/v-iashin/MDVC.
Blurry Video Frame Interpolation

Existing works reduce motion blur and up-convert frame rate through two separate ways, including frame deblurring and frame interpolation. However, few studies have approached the joint video enhancement problem, namely synthesizing high-frame-rate clear results from low-frame-rate blurry inputs. In this paper, we propose a blurry video frame interpolation method to reduce motion blur and up-convert frame rate simultaneously. Specifically, we develop a pyramid module to cyclically synthesize clear intermediate frames. The pyramid module features adjustable spatial receptive field and temporal scope, thus contributing to controllable computational complexity and restoration ability. Besides, we propose an inter-pyramid recurrent module to connect sequential models to exploit the temporal relationship. The pyramid module integrates a recurrent module, thus can iteratively synthesize temporally smooth results without significantly increasing the model size. Extensive experimental results demonstrate that our method performs favorably against state-of-the-art methods. The source code and pre-trained model are available at https://github.com/laomao0/BIN.
A Stochastic Conditioning Scheme for Diverse Human Motion Prediction

Human motion prediction, the task of predicting future 3D human poses given a sequence of observed ones, has been mostly treated as a deterministic problem. However, human motion is a stochastic process: Given an observed sequence of poses, multiple future motions are plausible. Existing approaches to modeling this stochasticity typically combine a random noise vector with information about the previous poses. This combination, however, is done in a deterministic manner, which gives the network the flexibility to learn to ignore the random noise. Alternatively, in this paper, we propose to stochastically combine the root of variations with previous pose information, so as to force the model to take the noise into account. We exploit this idea for motion prediction by incorporating it into a recurrent encoder-decoder network with a conditional variational autoencoder block that learns to exploit the perturbations. Our experiments on two large-scale motion prediction datasets demonstrate that our model yields high-quality pose sequences that are much more diverse than those from state-of-the-art stochastic motion prediction techniques.
How Useful Is Self-Supervised Pretraining for Visual Tasks?

Recent advances have spurred incredible progress in self-supervised pretraining for vision. We investigate what factors may play a role in the utility of these pretraining methods for practitioners. To do this, we evaluate various self-supervised algorithms across a comprehensive array of synthetic datasets and downstream tasks. We prepare a suite of synthetic data that enables an endless supply of annotated images as well as full control over dataset difficulty. Our experiments offer insights into how the utility of self-supervision changes as the number of available labels grows as well as how the utility changes as a function of the downstream task and the properties of the training data. We also find that linear evaluation does not correlate with finetuning performance. Code and data is available at github.com/princeton-vl/selfstudy.
Structure Boundary Preserving Segmentation for Medical Image With Ambiguous Boundary

In this paper, we propose a novel image segmentation method to tackle two critical problems of medical image, which are (i) ambiguity of structure boundary in the medical image domain and (ii) uncertainty of the segmented region without specialized domain knowledge. To solve those two problems in automatic medical segmentation, we propose a novel structure boundary preserving segmentation framework. To this end, the boundary key point selection algorithm is proposed. In the proposed algorithm, the key points on the structural boundary of the target object are estimated. Then, a boundary preserving block (BPB) with the boundary key point map is applied for predicting the structure boundary of the target object. Further, for embedding experts' knowledge in the fully automatic segmentation, we propose a novel shape boundary-aware evaluator (SBE) with the ground-truth structure information indicated by experts. The proposed SBE could give feedback to the segmentation network based on the structure boundary key point. The proposed method is general and flexible enough to be built on top of any deep learning-based segmentation network. We demonstrate that the proposed method could surpass the state-of-the-art segmentation network and improve the accuracy of three different segmentation network models on different types of medical image datasets.
COVID-19 pandemic reveals the peril of ignoring metadata standards

Efficient response to the pandemic through the mobilization of the larger scientific community is challenged by the limited reusability of the available primary genomic data. Here, the Genomic Standards Consortium board highlights the essential need for contextual genomic data FAIRness, for empowering key data-driven biological questions.
Heterogeneous comprehensive learning and dynamic multi-swarm particle swarm optimizer with two mutation operators

In this paper, a heterogeneous comprehensive learning and dynamic multi-swarm particle swarm optimizer with two mutation operators (HCLDMS-PSO) is presented. In addition, a comprehensive learning (CL) strategy with the global optimal experience of the whole population is conducted to generate an exploitation subpopulation exemplar. However, a modified dynamic multi-swarm (DMS) strategy is specially designed to construct the exploration subpopulation exemplar. In the canonical DMS strategy, it is unfavorable for different sub-swarms to use the same linear decreasing inertia weight parameter. We first propose classifying the DMS sub-swarms at the search level and then constructing a novel nonlinear adaptive decreasing inertia weight for different sub-swarms, introducing a non-uniform mutation operator to enhance its exploration capability. Finally, the gbest of the whole population also adopts a Gaussian mutation operator to avoid falling into the local optimum. The particles of the two subpopulations will update their velocity independently without crippling one another to prevent a loss of diversity. The performance of HCLDMS-PSO is compared with those of 8 other PSO variants and 11 evolutionary algorithms on two classical benchmark optimization problems and a real-world engineering problem. Experimental results demonstrate that the HCLDMS-PSO improves the convergence speed, accuracy, and reliability on most optimization problems.
Non-immersive virtual reality technologies in real estate: How customer experience drives attitudes toward properties and the service provider

Although previous research has highlighted that virtual reality (VR) technologies can enhance customer experience, the efficacy of VR technologies in real estate remains unclear. Therefore, this study examines the effects of offering consumers a non-immersive VR experience via a widely available technology, that allows them to view real estate products. Participants accessed the website of a real estate agency that presented apartments through either static photos or interactive 360° visits. The latter condition was associated with better “visiting” experiences and more positive attitudes toward both the products and the agency. This study highlights the advantages of such technologies in enhancing customer experience and attitudes, particularly in the highly competitive real estate industry.
A Novel Hybrid Cryptosystem for Secure Streaming of High Efficiency H.265 Compressed Videos in IoT Multimedia Applications

In this modernistic age of innovative technologies like big data processing, cloud computing, and Internet of things, the utilization of multimedia information is growing daily. In contrast to other forms of multimedia, videos are extensively utilized and streamed over the Internet and communication networks in numerous Internet of Multimedia Things (IoMT) applications. Consequently, there is an immense necessity to achieve secure video transmission over modern communication networks due to the third-party exploitation and falsification of transmitted and stored digital multimedia data. The present methods for secure communication of multimedia content between clouds and mobile devices have constraints in terms of processing load, memory support, data size, and battery power. These methods are not the optimum solutions for large-sized multimedia content and are not appropriate for the restricted resources of mobile devices and clouds. The High-Efficiency Video Coding (HEVC) is the latest and modern video codec standard introduced for efficiently storing and streaming of high-resolution videos with suitable size and higher quality. In this paper, a novel hybrid cryptosystem combining DNA (Deoxyribonucleic Acid) sequences, Arnold chaotic map, and Mandelbrot sets is suggested for secure streaming of compressed HEVC streams. Firstly, the high-resolution videos are encoded using the H.265/HEVC codec to achieve efficient compression performance. Subsequently, the suggested Arnold chaotic map ciphering process is employed individually on three channels (Y, U, and V) of the compressed HEVC frame. Then, the DNA encoding sequences are established on the primary encrypted frames resulted from the previous chaotic ciphering process. After that, a modified Mandelbrot set-based conditional shift process is presented to effectively introduce confusion features on the Y, U, and V channels of the resulted ciphered frames. Massive simulation results and security analysis are performed to substantiate that the suggested HEVC cryptosystem reveals astonishing robustness and security accomplishment in contrast to the literature cryptosystems.
Conditional permutation importance revisited

Random forest based variable importance measures have become popular tools for assessing the contributions of the predictor variables in a fitted random forest. In this article we reconsider a frequently used variable importance measure, the Conditional Permutation Importance (CPI). We argue and illustrate that the CPI corresponds to a more partial quantification of variable importance and suggest several improvements in its methodology and implementation that enhance its practical value. In addition, we introduce the threshold value in the CPI algorithm as a parameter that can make the CPI more partial or more marginal.By means of extensive simulations, where the original version of the CPI is used as the reference, we examine the impact of the proposed methodological improvements. The simulation results show how the improved CPI methodology increases the interpretability and stability of the computations. In addition, the newly proposed implementation decreases the computation times drastically and is more widely applicable. The improved CPI algorithm is made freely available as an add-on package to the open-source software R.The proposed methodology and implementation of the CPI is computationally faster and leads to more stable results. It has a beneficial impact on practical research by making random forest analyses more interpretable.
GBDT-MO: Gradient-Boosted Decision Trees for Multiple Outputs

Gradient-boosted decision trees (GBDTs) are widely used in machine learning, and the output of current GBDT implementations is a single variable. When there are multiple outputs, GBDT constructs multiple trees corresponding to the output variables. The correlations between variables are ignored by such a strategy causing redundancy of the learned tree structures. In this article, we propose a general method to learn GBDT for multiple outputs, called GBDT-MO. Each leaf of GBDT-MO constructs predictions of all variables or a subset of automatically selected variables. This is achieved by considering the summation of objective gains over all output variables. Moreover, we extend histogram approximation into the multiple-output case to speed up training. Various experiments on synthetic and real data sets verify that GBDT-MO achieves outstanding performance in terms of accuracy, training speed, and inference speed.
A novel randomised particle swarm optimizer

The particle swarm optimization (PSO) algorithm is a popular evolutionary computation approach that has received an ever-increasing interest in the past decade owing to its wide application potential. Despite the many variants of the PSO algorithm with improved search ability by means of both the convergence rate and the population diversity, the local optima problem remains a major obstacle that hinders the global optima from being found. In this paper, a novel randomized particle swarm optimizer (RPSO) is proposed where the Gaussian white noise with adjustable intensity is utilized to randomly perturb the acceleration coefficients in order for the problem space to be explored more thoroughly. With this new strategy, the RPSO algorithm not only maintains the population diversity but also enhances the possibility of escaping the local optima trap. Experimental results demonstrate that the proposed RPSO algorithm outperforms some existing popular variants of PSO algorithms on a series of widely used optimization benchmark functions.
A Survey of Multi-Task Deep Reinforcement Learning

Driven by the recent technological advancements within the field of artificial intelligence research, deep learning has emerged as a promising representation learning technique across all of the machine learning classes, especially within the reinforcement learning arena. This new direction has given rise to the evolution of a new technological domain named deep reinforcement learning, which combines the representational learning power of deep learning with existing reinforcement learning methods. Undoubtedly, the inception of deep reinforcement learning has played a vital role in optimizing the performance of reinforcement learning-based intelligent agents with model-free based approaches. Although these methods could improve the performance of agents to a greater extent, they were mainly limited to systems that adopted reinforcement learning algorithms focused on learning a single task. At the same moment, the aforementioned approach was found to be relatively data-inefficient, particularly when reinforcement learning agents needed to interact with more complex and rich data environments. This is primarily due to the limited applicability of deep reinforcement learning algorithms to many scenarios across related tasks from the same environment. The objective of this paper is to survey the research challenges associated with multi-tasking within the deep reinforcement arena and present the state-of-the-art approaches by comparing and contrasting recent solutions, namely DISTRAL (DIStill &amp; TRAnsfer Learning), IMPALA(Importance Weighted Actor-Learner Architecture) and PopArt that aim to address core challenges such as scalability, distraction dilemma, partial observability, catastrophic forgetting and negative knowledge transfer.
NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size

Neural architecture search (NAS) has attracted a lot of attention and has been illustrated to bring tangible benefits in a large number of applications in the past few years. Architecture topology and architecture size have been regarded as two of the most important aspects for the performance of deep learning models and the community has spawned lots of searching algorithms for both of those aspects of the neural architectures. However, the performance gain from these searching algorithms is achieved under different search spaces and training setups. This makes the overall performance of the algorithms incomparable and the improvement from a sub-module of the searching model unclear. In this paper, we propose NATS-Bench, a unified benchmark on searching for both topology and size, for (almost) any up-to-date NAS algorithm. NATS-Bench includes the search space of 15,625 neural cell candidates for architecture topology and 32,768 for architecture size on three datasets. We analyze the validity of our benchmark in terms of various criteria and performance comparison of all candidates in the search space. We also show the versatility of NATS-Bench by benchmarking 13 recent state-of-the-art NAS algorithms on it. All logs and diagnostic information trained using the same setup for each candidate are provided. This facilitates a much larger community of researchers to focus on developing better NAS algorithms in a more comparable and computationally effective environment. All codes are publicly available at: https://xuanyidong.com/assets/projects/NATS-Bench.