Reviewerly/exHarmony huggingface dataset:
Searching Privately by Imperceptible Lying: A Novel Private Hashing Method with Differential Privacy

In the big data era, with the increasing amount of multi-media data, approximate nearest neighbor~(ANN) search has been an important but challenging problem. As a widely applied large-scale ANN search method, hashing has made great progress, and achieved sub-linear search time with low memory space. However, the advances in hashing are based on the availability of large and representative datasets, which often contain sensitive information. Typically, the privacy of this individually sensitive information is compromised. In this paper, we tackle this valuable yet challenging problem and formulate a task termed as private hashing, which takes into account both searching performance and privacy protection. Specifically, we propose a novel noise mechanism, i.e., Random Flipping, and two private hashing algorithms, i.e., PHashing and PITQ, with the refined analysis within the framework of differential privacy, since differential privacy is a well-established technique to measure the privacy leakage of an algorithm. Random Flipping targets binary scenarios and leverages the "Imperceptible Lying" idea to guarantee ε-differential privacy by flipping each datum of the binary matrix (noise addition). To preserve ε-differential privacy, PHashing perturbs and adds noise to the hash codes learned by non-private hashing algorithms using Random Flipping. However, the noise addition for privacy in PHashing will cause severe performance drops. To alleviate this problem, PITQ leverages the power of alternative learning to distribute the noise generated by Random Flipping into each iteration while preserving ε-differential privacy. Furthermore, to empirically evaluate our algorithms, we conduct comprehensive experiments on the image search task and demonstrate that proposed algorithms achieve equal performance compared with non-private hashing methods.
Reproducibility Companion Paper: Visual Sentiment Analysis for Review Images with Item-Oriented and User-Oriented CNN

We revisit our contributions on visual sentiment analysis for online review images published at ACM Multimedia 2017, where we develop item-oriented and user-oriented convolutional neural networks that better capture the interaction of image features with specific expressions of users or items. In this work, we outline the experimental claims as well as describe the procedures to reproduce the results therein. In addition, we provide artifacts including data sets and code to replicate the experiments.
Semantics Preserving Hierarchy based Retrieval of Indian heritage monuments

Monument classification can be performed on the basis of their appearance and shape from coarse to fine categories, although there is much semantic information present in the monuments which is reflected in the eras they were built, its type or purpose, the dynasty which established it, etc. Particularly, Indian subcontinent exhibits a huge deal of variation in terms of architectural styles owing to its rich cultural heritage. In this paper, we propose a framework that utilizes hierarchy to preserve semantic information while performing image classification or image retrieval. We encode the learnt deep semantic embeddings to construct a dictionary of images and then utilize a re-ranking framework on the the retrieved results using DeLF features. The semantic information preserved in these embeddings helps to classify unknown monuments at higher level of granularity in hierarchy. We have curated a large, novel Indian heritage monuments dataset (IHMD) comprising of images of historical, cultural and religious importance with subtypes of eras, dynasties and architectural styles. We demonstrate the performance of the proposed framework in image classification and retrieval tasks and compare it with other competing methods on this dataset. The dataset can be accessed at: https://github.com/ronakgupta/HierarchicalSemanticsEmbeddings.
The Power of Communities: A Text Classification Model with Automated Labeling Process Using Network Community Detection

Text classification is one of the most critical areas in machine learning and artificial intelligence research. It has been actively adopted in many business applications such as conversational intelligence systems, news articles categorizations, sentiment analysis, emotion detection systems, and many other recommendation systems in our daily life. One of the problems in supervised text classification models is that the models’ performance depends heavily on the quality of data labeling that is typically done by humans. In this study, we propose a new network community detection-based approach to automatically label and classify text data into multiclass value spaces. Specifically, we build networks with sentences as the network nodes and pairwise cosine similarities between the Term Frequency-Inversed Document Frequency (TFIDF) vector representations of the sentences as the network link weights. We use the Louvain method to detect the communities in the sentence networks. We train and test the Support Vector Machine and the Random Forest models on both the human-labeled data and network community detection labeled data. Results showed that models with the data labeled by the network community detection outperformed the models with the human-labeled data by 2.68–3.75% of classification accuracy. Our method may help developments of more accurate conversational intelligence and other text classification systems.
Single Image De-noising via Staged Memory Network

Single image de-noising is an important yet under-explored task to estimate the underlying clean image from its noisy observation. It poses great challenges over the balance between over-de-noising (e.g., mistakenly remove texture details in noise-free regions) and under-de-noising (e.g., leave noisy points). Existing works solely treat the removal of noise from images as a process of pixel-wise regression and lack of preserving image details. In this paper, we firstly propose a Staged Memory Network (SMNet) consisting of noise memory stage and image memory stage for explicitly exploring the staged memories of our network in single image de-noising with different noise levels. Specifically, the noise memory stage is to reveal noise characteristics by using local-global spatial dependencies via an encoder-decoder sub-network composed of dense blocks and noise-aware blocks. Taking the residual result between the input noise image and the prediction of the noise memory stage as input, the image memory stage continues to get a noise-free and well-reconstructed output image via a contextual fusion sub-network with contextual blocks and a fusion block. Solid and comprehensive experiments on three tasks (i.e. synthetic and real data, and blind de-noising) demonstrate that our SMNet can significantly achieve better performance compared with state-of-the-art methods by cleaning noisy images with various densities, scales and intensities while keeping the image details of noise-free regions well-preserved. Moreover, interpretability analysis is added to further prove the ability of our composed memory stages.
Analysis of Multitasking Evolutionary Algorithms under the Order of Solution Variables

Recently, it was demonstrated that multitasking evolutionary algorithm (MTEA), a newly proposed algorithm, can solve multiple optimization problems simultaneously through a single run, breaking through the limitations of traditional evolutionary algorithms (EAs), with good convergence and exploration performance. As a novel algorithm, MTEA still has a lot of unexplored space. Generally speaking, the order of solution variables has no significant influence on the single-tasking EAs. To our knowledge, the effect of the order of variables in the multitasking scenario has not been explored. To fill in this research gap, three orders of variables in the multitasking scenario are proposed in this paper, including full reverse order, bisection reverse order, and trisection reverse order. An important feature of these orders of variables is that an individual can recover as himself after two times of changing the order of variables. In order to verify our idea, these orders of variables are embedded into MTEA. The experiment results revealed that the effect of the different orders of variables is universal but not significant enough in the practical application. Furthermore, tasks with high similarity and high degree of intersection are sensitive to the order of variables and get great impact between tasks.
Audio Adversarial Examples for Robust Hybrid CTC/Attention Speech Recognition

Recent advances in Automatic Speech Recognition (ASR) demonstrated how end-to-end systems are able to achieve state-of-the-art performance. There is a trend towards deeper neural networks, however those ASR models are also more complex and prone against specially crafted noisy data. Those Audio Adversarial Examples (AAE) were previously demonstrated on ASR systems that use Connectionist Temporal Classification (CTC), as well as attention-based encoder-decoder architectures. Following the idea of the hybrid CTC/attention ASR system, this work proposes algorithms to generate AAEs to combine both approaches into a joint CTC-attention gradient method. Evaluation is performed using a hybrid CTC/attention end-to-end ASR model on two reference sentences as case study, as well as the TEDlium v2 speech recognition task. We then demonstrate the application of this algorithm for adversarial training to obtain a more robust ASR model.
An improved genetic algorithm for mobile robot path planning in grid environment

This paper proposes an improved genetic algorithm to study the path planning of mobile robot in grid environment. First, rasterize the motion plane of the robot, use serial number coding method and design a heuristic median insertion method to establish the initial population, ensure that the planned initial paths are all feasible paths, thereby speeding up the convergence of the algorithm. Then assign different weights to the path length, path security, and path energy consumption and combine them to generate a multi-objective fitness function. Finally, improve some genetic operations to maintain the population diversity of the algorithm in the later period, and avoid the algorithm from falling into avoid premature. Simulation experiments show that the proposed algorithm can quickly plan a feasible path in the grid environment. The path is not only shorter in length, but also more stable. At the same time, the running speed of the algorithm is 45.7% higher than other improved algorithms.
Constructing a Maximally Entangled Seven-Qubit State via Orthogonal Arrays

Huber et al. [Phys. Rev. Lett. 118 (2017) 200502] have proved that a seven-qubit state whose three-body marginal states are all maximally mixed does not exist. Here, we propose a method to build a maximally entangled state based on orthogonal arrays to construct maximally entangled seven-qubit states. Using this method, we not only determine that a seven-qubit state whose three-body marginals are all maximally mixed does not exist, but also find the condition for maximally entangled seven-qubit states. We consider that π ME = 19/140 is a condition for maximally entangled seven-qubit states. Furthermore, we derive three forms of maximally entangled seven-qubit states via orthogonal arrays.
A personalized event–participant arrangement framework based on user interests in social network

In recent years, online event-based social networks (EBSNs), such as Meetup and Whova, which offer platforms for users to plan, publish and arrange events, have gained popularity and rapid development. A fundamental task of managing EBSNs is to recommend a personalized and suitable sequence of events for potential users. However, none of the existing approaches consider both personalized recommendation and event arrangement at the same time. In addition, as to event-to-user recommendation, there are many existing methods, but they neglect individual differences between users and the differences between event sets that users can participate in. As to event arrangement, the different requirements of service providers are ignored, such as time efficiency, user experience, etc. In order to address these problems, in this paper, we first define the problem of Personalized Event–Participant Arrangement based on user interests (PEPA), which contains two parts: personalized recommendation and event–participant arrangement. For the personalized recommendation, we propose a personalized recommendation framework, which treats each user differently and computes the interest value toward events for each user, and we put forward a concept of user curiosity for the first time to overcome the problem of neglecting the differences between event sets for each user. For the event arrangement, we devise a Pruning-based greedy algorithm and two improvement strategies called Conflict-set strategy and Relaxation strategy to improve the runtime efficiency and the user experience respectively. Finally, we conduct extensive experiments on real and synthetic datasets to verify the efficiency and the user satisfaction of the proposed algorithms.
Hybrid Human-Machine Classification System for Cultural Heritage Data

The advancement of digital technologies has helped cultural heritage organizations to digitize their data collections and improve the accessibility via online platforms. These platforms have enabled citizens to contribute to the process of digital preservation of cultural heritage by sharing documents and their knowledge. However, many historical datasets have problems due to incomplete metadata. To solve this issue, cultural heritage organizations heavily depend on domain experts. In this paper, we address the issue of completing the metadata of historical digital collections. For this, we introduce a new hybrid human-machine model. This model jointly integrates predictions of a deep multi-input model and inferred labels from multiple crowd judgements. The multi-input model uses visual features extracted from the images and textual features from the metadata, complemented with Wikipedia classes of concepts extracted in the text. On the crowd answer aggregation, our method considers the workers' reliability scores. This score is based on the performance of workers' task history and their performance in our task. We have applied our hybrid approach to a culture heritage platform and the evaluations show that it outperforms both deep learning and crowdsourcing when applied individually.
A Generative Modeling Approach for Interpreting Population-Level Variability in Brain Structure

Understanding how neural structure varies across individuals is critical for characterizing the effects of disease, learning, and aging on the brain. However, disentangling the different factors that give rise to individual variability is still an outstanding challenge. In this paper, we introduce a deep generative modeling approach to find different modes of variation across many individuals. Our approach starts with training a variational autoencoder on a collection of auto-fluorescence images from a little over 1,700 mouse brains at 25 \(\upmu \)m resolution. We then tap into the learned factors and validate the model’s expressiveness, via a novel bi-directional technique that makes structured perturbations to both, the high-dimensional inputs of the network, as well as the low-dimensional latent variables in its bottleneck. Our results demonstrate that through coupling generative modeling frameworks with structured perturbations, it is possible to probe the latent space of the generative model to provide insights into the representations of brain structure formed in deep networks.
Learning Global Structure Consistency for Robust Object Tracking

Fast appearance variations and the distractions of similar objects are two of the most challenging problems in visual object tracking. Unlike many existing trackers that focus on modeling only the target, in this work, we consider the transient variations of the whole scene. The key insight is that the object correspondence and spatial layout of the whole scene are consistent (i.e., global structure consistency) in consecutive frames which helps to disambiguate the target from distractors. Moreover, modeling transient variations enables to localize the target under fast variations. Specifically, we propose an effective and efficient short-term model that learns to exploit the global structure consistency in a short time and thus can handle fast variations and distractors. Since short-term modeling falls short of handling occlusion and out of the views, we adopt the long-short term paradigm and use a long-term model that corrects the short-term model when it drifts away from the target or the target is not present. These two components are carefully combined to achieve the balance of stability and plasticity during tracking. We empirically verify that the proposed tracker can tackle the two challenging scenarios and validate it on large scale benchmarks. Remarkably, our tracker improves state-of-the-art-performance on VOT2018 from 0.440 to 0.460, GOT-10k from 0.611 to 0.640, and NFS from 0.619 to 0.629.
Hybrid powered intelligent irrigation system using Oman Falaj and solar energy

The overall development of the agriculture sector will play a vital role in contributing to the Oman economy. Agriculture development depends on proper water utilization and its preservation from wastage. And solar radiation receives across the Sultanate of Oman is tremendous and highest in the world. Consider these two factors as the major potential to modernize and develop a sustainable model of irrigation. The use of modern technology is quite necessary and application of 4th industrial technologies such as AI-the Internet of things has stronger capabilities to solve this problem. This paper presents the construction and implementation of an intelligent irrigation system powered using the Oman falaj (hydro) along with solar. With the focus of better usage of falaj water with less wastage, and in turn, make use of the free energy available in the falaj and solar. The first module is hybrid power generation systems is develop using the turgo-hydro generator and PV panel. The energy generated from the hybrid system is used to charge the battery. The second module to develop a low-cost intelligent irrigation system. The design intelligent irrigation system consists of an Arduino based controller, sensor, transmitters, receivers, and other wireless subsystems. The measured variables from the cultivation land such as temperature, humidity, and moisture are transmitted by sensor to the Blynk IoT platform. The experimental system results show quite important outcomes in terms of an efficient way to use falaj water and its energy coupled with solar energy. Sustainable model of intelligent control irrigation and better utilization of the renewable energy available in Oman can be established.
Evolutionary optimization of Drone Trajectories Based on Optimal Reciprocal Collision Avoidance

In recent years, the advent of new hardware and software technologies for navigation and control has made Unmanned Aerial Vehicles (UAVs) ever more autonomous and efficient. As a consequence, it is now possible to have drones moving within complex environments, such as cities or indoor areas. One of the main requirements for intelligent mission planning in such environments is the ability to correctly and efficiently detect and avoid obstacles. For this reason, various libraries have been created for the simulation of UAV navigation in virtual environments, in order to test algorithms for automatic obstacle detection and collision avoidance before deploying the drones in the real world. Usually, the performance of these algorithms depends on various parameters as well as specific application settings. However, while different parameter configurations can be easily tested in simulation, their number can be too large to allow a complete exploration of the parameter space or a manual tuning. Furthermore, a full analytical model of the parameters' influence on the algorithmic performance can be hard to obtain. Yet, it is extremely important to find their optimal values to allow collision-free navigation. In this direction, we propose here a thorough exploration, based on an Evolutionary Algorithm (EA), of the parameter space of the Optimal Reciprocal Collision Avoidance (ORCA) algorithm. Our results show that the proposed EA is a viable solution for finding optimal parameter settings that can be generalizable to different scenarios characterized by different complexity levels.
Identifying Wrongly Predicted Samples: A Method for Active Learning

While unlabelled data can be largely available and even abundant, the annotation process can be quite expensive and limiting. Under the assumption that some samples are more important for a given task than others, active learning targets the problem of identifying the most informative samples that one should acquire annotations for. In this work we propose a simple sample selection criterion that moves beyond the conventional reliance on model uncertainty as proxy to leverage new labels. By first accepting the model prediction and then judging its effect on the generalization error, we can better identify wrongly predicted samples. We also present a very efficient approximation to our criterion, providing a similarity-based interpretation. In addition to evaluating our method on the standard benchmarks of active learning, we consider the challenging yet realistic imbalanced data scenario. We show state-of-the-art results, especially on the imbalanced setting, and achieve better rates at identifying wrongly predicted samples than existing active learning methods. Our method is simple, model agnostic and relies on the current model status without the need for re-training from scratch.
Learning From Music to Visual Storytelling of Shots: A Deep Interactive Learning Mechanism

Learning from music to visual storytelling of shots is an interesting and emerging task. It produces a coherent visual story in the form of a shot type sequence, which not only expands the storytelling potential for a song but also facilitates automatic concert video mashup process and storyboard generation. In this study, we present a deep interactive learning (DIL) mechanism for building a compact yet accurate sequence-to-sequence model to accomplish the task. Different from the one-way transfer between a pre-trained teacher network (or ensemble network) and a student network in knowledge distillation (KD), the proposed method enables collaborative learning between an ensemble teacher network and a student network. Namely, the student network also teaches. Specifically, our method first learns a teacher network that is composed of several assistant networks to generate a shot type sequence and produce the soft target (shot types) distribution accordingly through KD. It then constructs the student network that learns from both the ground truth label (hard target) and the soft target distribution to alleviate the difficulty of optimization and improve generalization capability. As the student network gradually advances, it turns to feed back knowledge to the assistant networks, thereby improving the teacher network in each iteration. Owing to such interactive designs, the DIL mechanism bridges the gap between the teacher and student networks and produces more superior capability for both networks. Objective and subjective experimental results demonstrate that both the teacher and student networks can generate more attractive shot sequences from music, thereby enhancing the viewing and listening experience.