Reviewerly/exHarmony huggingface dataset:
Enforcing opacity by insertion functions under multiple energy constraints

This paper investigates the enforcement of opacity by insertion functions subject to multiple quantitative constraints capturing resource or energy limitations. There is a malicious intruder attempting to infer secrets of the system from its observations. To prevent the disclosure of secrets, the insertion function inserts fictitious events to the output of the system to obfuscate the intruder. The system is initialized with several types of resources, referred to as energy. The energy is consumed or replenished with event occurrences while always consumed with event insertions. The insertion function must enforce opacity while ensuring that each type of resource is never depleted. This problem is then reduced to a two-player game between the insertion function and the system (environment), with properly defined objectives. A game structure called the Energy Insertion Structure, denoted by EIS is proposed, which provably contains solutions to the energy constrained opacity enforcement problem. Then we further study the bounded cost rate insertion problem on the insertion function's winning region of EIS, which requires that the long run average rate of insertion cost be bounded. This problem is formulated as a multidimensional mean payoff game and a special method called hyperplane separation technique is applied to efficiently solve it.
Fast and Effective Global Covariance Pooling Network for Image Steganalysis

Recently, deep learning based methods have achieved superior performance compared to conventional methods based on hand-crafted features in image steganalysis. However, most modern methods are usually quite time consuming. For instance, it takes over 3 days to train a state-of-the-art neural network, i.e. SRNet [3] in our experiments. In this paper, therefore, we propose a fast yet very effective convolutional neural network (CNN) for image steganalysis in spatial domain. To make a good tradeoff between training time and performance, we carefully design the architecture of the proposed network according to our extensive experiments. In addition, we first introduce the global covariance pooling into steganalysis to exploit the second-order statistic of high-level features for further improving the performance. Experimental results show that the proposed network can outperform the current best one, while its training time is significantly reduced.
BOEW: A Content-Based Image Retrieval Scheme Using Bag-of-Encrypted-Words in Cloud Computing

Content-based Image Retrieval (CBIR) techniques have been extensively studied with the rapid growth of digital images. Generally, CBIR service is quite expensive in computational and storage resources. Thus, it is a good choice to outsource CBIR service to the cloud server that is equipped with enormous resources. However, the privacy protection becomes a big problem, as the cloud server cannot be fully trusted. In this paper, we propose an outsourced CBIR scheme based on a novel bag-of-encrypted-words (BOEW) model. The image is encrypted by color value substitution, block permutation, and intra-block pixel permutation. Then, the local histograms are calculated from the encrypted image blocks by the cloud server. All the local histograms are clustered together, and the cluster centers are used as the encrypted visual words. In this way, the bag-of-encrypted-words (BOEW) model is built to represent each image by a feature vector, i.e., a normalized histogram of the encrypted visual words. The similarity between images can be directly measured by the Manhattan distance between feature vectors on the cloud server side. Experimental results and security analysis on the proposed scheme demonstrate its search accuracy and security.
The capacity of feedforward neural networks

A long standing open problem in the theory of neural networks is the development of quantitative methods to estimate and compare the capabilities of different architectures. Here we define the capacity of an architecture by the binary logarithm of the number of functions it can compute, as the synaptic weights are varied. The capacity provides an upperbound on the number of bits that can be extracted from the training data and stored in the architecture during learning. We study the capacity of layered, fully-connected, architectures of linear threshold neurons with L layers of size n1,n2,…,nL and show that in essence the capacity is given by a cubic polynomial in the layer sizes: C(n1,…,nL)=∑k=1L-1min(n1,…,nk)nknk+1, where layers that are smaller than all previous layers act as bottlenecks. In proving the main result, we also develop new techniques (multiplexing, enrichment, and stacking) as well as new bounds on the capacity of finite sets. We use the main result to identify architectures with maximal or minimal capacity under a number of natural constraints. This leads to the notion of structural regularization for deep architectures. While in general, everything else being equal, shallow networks compute more functions than deep networks, the functions computed by deep networks are more regular and "interesting".
Facial Emotion Distribution Learning by Exploiting Low-Rank Label Correlations Locally

Emotion recognition from facial expressions is an interesting and challenging problem and has attracted much attention in recent years. Substantial previous research has only been able to address the ambiguity of “what describes the expression”, which assumes that each facial expression is associated with one or more predefined affective labels while ignoring the fact that multiple emotions always have different intensities in a single picture. Therefore, to depict facial expressions more accurately, this paper adopts a label distribution learning approach for emotion recognition that can address the ambiguity of “how to describe the expression” and proposes an emotion distribution learning method that exploits label correlations locally. Moreover, a local low-rank structure is employed to capture the local label correlations implicitly. Experiments on benchmark facial expression datasets demonstrate that our method can better address the emotion distribution recognition problem than state-of-the-art methods.
Truly Unsupervised Acoustic Word Embeddings Using Weak Top-down Constraints in Encoder-decoder Models

We investigate unsupervised models that can map a variable-duration speech segment to a fixed-dimensional representation. In settings where unlabelled speech is the only available resource, such acoustic word embeddings can form the basis for "zero-resource" speech search, discovery and indexing systems. Most existing unsupervised embedding methods still use some supervision, such as word or phoneme boundaries. Here we propose the encoder-decoder correspondence autoencoder (EncDec-CAE), which, instead of true word segments, uses automatically discovered segments: an unsupervised term discovery system finds pairs of words of the same unknown type, and the EncDec-CAE is trained to reconstruct one word given the other as input. We compare it to a standard encoder-decoder autoencoder (AE), a variational AE with a prior over its latent embedding, and downsampling. EncDec-CAE outperforms its closest competitor by 29% relative in average precision on two languages in a word discrimination task.
Benchmark Data and Method for Real-Time People Counting in Cluttered Scenes Using Depth Sensors

Vision-based automatic counting of people has widespread applications in intelligent transportation systems, security, and logistics. However, there is currently no large-scale public dataset for benchmarking approaches on this problem. This paper fills this gap by introducing the first real-world RGB-D people counting dataset (PCDS) containing over 4500 videos recorded at the entrance doors of buses in normal and cluttered conditions. It also proposes an efficient method for counting people in real-world cluttered scenes related to public transportations using depth videos. The proposed method computes a point cloud from the depth video frame and re-projects it onto the ground plane to normalize the depth information. The resulting depth image is analyzed for identifying potential human heads. The human head proposals are meticulously refined using a 3D human model. The proposals in each frame of the continuous video stream are tracked to trace their trajectories. The trajectories are again refined to ascertain reliable counting. People are eventually counted by accumulating the head trajectories leaving the scene. To enable effective head and trajectory identification, we also propose two different compound features. A thorough evaluation on PCDS demonstrates that our technique is able to count people in cluttered scenes with high accuracy at 45 fps on a 1.7-GHz processor, and hence it can be deployed for effective real-time people counting for intelligent transportation systems.
Recurrent Saliency Transformation Network for Tiny Target Segmentation in Abdominal CT Scans

We aim at segmenting a wide variety of organs, including tiny targets (e.g., adrenal gland), and neoplasms (e.g., pancreatic cyst), from abdominal CT scans. This is a challenging task in two aspects. First, some organs (e.g., the pancreas), are highly variable in both anatomy and geometry, and thus very difficult to depict. Second, the neoplasms often vary a lot in its size, shape, as well as its location within the organ. Third, the targets (organs and neoplasms) can be considerably small compared to the human body, and so standard deep networks for segmentation are often less sensitive to these targets and thus predict less accurately especially around their boundaries. In this paper, we present an end-to-end framework named recurrent saliency transformation network (RSTN) for segmenting tiny and/or variable targets. The RSTN is a coarse-to-fine approach that uses prediction from the first (coarse) stage to shrink the input region for the second (fine) stage. A saliency transformation module is inserted between these two stages so that 1) the coarse-scaled segmentation mask can be transferred as spatial weights and applied to the fine stage and 2) the gradients can be back-propagated from the loss layer to the entire network so that the two stages are optimized in a joint manner. In the testing stage, we perform segmentation iteratively to improve accuracy. In this extended journal paper, we allow a gradual optimization to improve the stability of the RSTN, and introduce a hierarchical version named H-RSTN to segment tiny and variable neoplasms such as pancreatic cysts. Experiments are performed on several CT datasets including a public pancreas segmentation dataset, our own multi-organ dataset, and a cystic pancreas dataset. In all these cases, the RSTN outperforms the baseline (a stage-wise coarse-to-fine approach) significantly. Confirmed by the radiologists in our team, these promising segmentation results can help early diagnosis of pancreatic cancer. The code and pre-trained models of our project were made available at https://github.com/198808xc/OrganSegRSTN.
Assessing the accuracy and teachers' impressions of Google Translate: A study of primary L2 writers in Hong Kong

• Primary 6 students wrote on the same topic in English and their native Chinese. • The Chinese scripts were translated into English using Google Translate (GT). • 12 teachers (without knowing about GT) graded the two sets of scripts. • The GT-translated scripts were graded equal or higher. • Teachers were interviewed for their impressions of GT.
Homomorphic Time-Lock Puzzles and Applications

Time-lock puzzles allow one to encrypt messages for the future, by efficiently generating a puzzle with a solution s that remains hidden until time $$\mathcal {T}$$ has elapsed. The solution is required to be concealed from the eyes of any algorithm running in (parallel) time less than $$\mathcal {T}$$ . We put forth the concept of homomorphic time-lock puzzles, where one can evaluate functions over puzzles without solving them, i.e., one can manipulate a set of puzzles with solutions $$(s_1, \dots , s_n)$$ to obtain a puzzle that solves to $$f(s_1, \ldots , s_n)$$ , for any function f. We propose candidate constructions under concrete cryptographic assumptions for different classes of functions. Then we show how homomorphic time-lock puzzles overcome the limitations of classical time-lock puzzles by proposing new protocols for applications of interest, such as e-voting, multi-party coin flipping, and fair contract signing.
Exploring the frontiers of eye tracking research in language studies: a novel co-citation scientometric review

Eye tracking technology has become an increasingly popular methodology in language studies. Using data from 27 journals in language sciences indexed in the Social Science Citation Index and/or Scopus, we conducted an in-depth scientometric analysis of 341 research publications together with their 14,866 references between 1994 and 2018. We identified a number of countries, researchers, universities, and institutes with large numbers of publications in eye tracking research in language studies. We further discovered a mixed multitude of connected research trends that have shaped the nature and development of eye tracking research. Specifically, a document co-citation analysis revealed a number of major research clusters, their key topics, connections, and bursts (sudden citation surges). For example, the foci of clusters #0 through #5 were found to be perceptual learning, regressive eye movement(s), attributive adjective(s), stereotypical gender, discourse processing, and bilingual adult(s). The content of all the major clusters was closely examined and synthesized in the form of an in-depth review. Finally, we grounded the findings within a data-driven theory of scientific revolution and discussed how the observed patterns have contributed to the emergence of new trends. As the first scientometric investigation of eye tracking research in language studies, the present study offers several implications for future research that are discussed.
Efficient Trajectory Planning for High Speed Flight in Unknown Environments

There has been considerable recent work in motion planning for UAVs to enable aggressive, highly dynamic flight in known environments with motion capture systems. However, these existing planners have not been shown to enable the same kind of flight in unknown, outdoor environments. In this paper we present a receding horizon planning architecture that enables the fast replanning necessary for reactive obstacle avoidance by combining three techniques. First, we show how previous work in computationally efficient, closed-form trajectory generation method can be coupled with spatial partitioning data structures to reason about the geometry of the environment in real-time. Second, we show how to maintain safety margins during fast flight in unknown environments by planning velocities according to obstacle density. Third, our receding-horizon, sampling-based motion planner uses minimum-jerk trajectories and closed-loop tracking to enable smooth, robust, high-speed flight with the low angular rates necessary for accurate visual-inertial navigation. We compare against two state-of-the-art, reactive motion planners in simulation and benchmark solution quality against an offline global planner. Finally, we demonstrate our planner over 80 flights with a combined distance of 22km of autonomous quadrotor flights in an urban environment at speeds up to 9.4ms $^{-1}$.
Decomposition for Large-scale Optimization Problems with Overlapping Components

In this paper we use a divide-and-conquer approach to tackle large-scale optimization problems with overlapping components. Decomposition for an overlapping problem is challenging as its components depend on one another. The existing decomposition methods typically assign all the linked decision variables into one group, thus cannot reduce the original problem size. To address this issue we modify the Recursive Differential Grouping (RDG) method to decompose overlapping problems, by breaking the linkage at variables shared by multiple components. To evaluate the efficacy of our method, we extend two existing overlapping benchmark problems considering various level of overlap. Experimental results show that our method can greatly improve the search ability of an optimization algorithm via divide-and-conquer, and outperforms RDG, random decomposition as well as other state-of-the-art methods. We further evaluate our method using the CEC’2013 benchmark problems and show that our method is very competitive when equipped with a component optimizer.
Multimodal feature fusion by relational reasoning and attention for visual question answering

The recently emerged research of Visual Question Answering (VQA) has become a hot topic in computer vision. A key solution to VQA exists in how to fuse multimodal features extracted from image and question. In this paper, we show that combining visual relationship and attention together achieves more fine-grained feature fusion. Specifically, we design an effective and efficient module to reason complex relationship between visual objects. In addition, a bilinear attention module is learned for question guided attention on visual objects, which allows us to obtain more discriminative visual features. Given an image and a question in natural language, our VQA model learns visual relational reasoning network and attention network in parallel to fuse fine-grained textual and visual features, so that answers can be predicted accurately. Experimental results show that our approach achieves new state-of-the-art performance of single model on both VQA 1.0 and VQA 2.0 datasets.
Deep Learning Based Code Smell Detection

Code smells are structures in the source code that suggest the possibility of refactorings. Consequently, developers may identify refactoring opportunities by detecting code smells. However, manual identification of code smells is challenging and tedious. To this end, a number of approaches have been proposed to identify code smells automatically or semi-automatically. Most of such approaches rely on manually designed heuristics to map manually selected source code metrics into predictions. However, it is challenging to manually select the best features. It is also difficult to manually construct the optimal heuristics. To this end, in this paper we propose a deep learning based novel approach to detecting code smells. The key insight is that deep neural networks and advanced deep learning techniques could automatically select features of source code for code smell detection, and could automatically build the complex mapping between such features and predictions. A big challenge for deep learning based smell detection is that deep learning often requires a large number of labeled training data (to tune a large number of parameters within the employed deep neural network) whereas existing datasets for code smell detection are rather small. To this end, we propose an automatic approach to generating labeled training data for the neural network based classifier, which does not require any human intervention. As an initial try, we apply the proposed approach to four common and well-known code smells, i.e., feature envy, long method, large class, and misplaced class. Evaluation results on open-source applications suggest that the proposed approach significantly improves the state-of-the-art.
Traffic Sign Detection Under Challenging Conditions: A Deeper Look into Performance Variations and Spectral Characteristics

Traffic signs are critical for maintaining the safety and efficiency of our roads. Therefore, we need to carefully assess the capabilities and limitations of automated traffic sign detection systems. Existing traffic sign datasets are limited in terms of type and severity of challenging conditions. Metadata corresponding to these conditions are unavailable and it is not possible to investigate the effect of a single factor because of the simultaneous changes in numerous conditions. To overcome the shortcomings in existing datasets, we introduced the CURE-TSD-Real dataset, which is based on simulated challenging conditions that correspond to adversaries that can occur in real-world environments and systems. We test the performance of two benchmark algorithms and show that severe conditions can result in an average performance degradation of 29% in precision and 68% in recall. We investigate the effect of challenging conditions through spectral analysis and show that the challenging conditions can lead to distinct magnitude spectrum characteristics. Moreover, we show that mean magnitude spectrum of changes in video sequences under challenging conditions can be an indicator of detection performance. The CURE-TSD-Real dataset is available online at https://github.com/olivesgatech/CURE-TSD.
Short-Range Radar-Based Gesture Recognition System Using 3D CNN With Triplet Loss

Gesture recognition is the most intuitive form of human computer-interface. Gesture sensing can replace interfaces such as touch and clicks needed for interacting with a device. Gesture recognition in a practical scenario is an open-set classification, i.e. the recognition system should classify correct known gestures while rejecting arbitrary unknown gestures during inference. To address the issue of gesture recognition in an open set, we present, in this paper, a novel distance-metric based meta-learning approach to learn embedding features from a video of range-Doppler images generated by hand gestures at the radar receiver. Further, k-Nearest Neighbor (kNN) is used to classify known gestures, distance-thresholding is used to reject unknown gesture motions and clustering is used to add new custom gestures on-the-fly without explicit model re-training. We propose to use 3D Deep Convolutional Neural Network (3D-DCNN) architecture to learn the embedding model using distance-based triplet-loss similarity metric. We demonstrate our approach to correctly classify gestures using short-range 60-GHz compact short-range radar sensor achieving an overall classification accuracy of 94.5% over six fine-grained gestures under challenging practical environments, while rejecting other unknown gestures with 0.935 F1 score, and capable of adding new gestures on-the-fly without an explicit model re-training.
Human-Centric AI for Trustworthy IoT Systems With Explainable Multilayer Perceptrons

Internet of Things (IoT) widely use analysis of data with artificial intelligence (AI) techniques in order to learn from user actions, support decisions, track relevant aspects of the user, and notify certain events when appropriate. However, most AI techniques are based on mathematical models that are difficult to understand by the general public, so most people use AI-based technology as a black box that they eventually start to trust based on their personal experience. This article proposes to go a step forward in the use of AI in IoT, and proposes a novel approach within the Human-centric AI field for generating explanations about the knowledge learned by a neural network (in particular a multilayer perceptron) from IoT environments. More concretely, this work proposes two techniques based on the analysis of artificial neuron weights, and another technique aimed at explaining each estimation based on the analysis of training cases. This approach has been illustrated in the context of a smart IoT kitchen that detects the user depression based on the food used for each meal, using a simulator for this purpose. The results revealed that most auto-generated explanations made sense in this context (i.e. 97.0%), and the execution times were low (i.e. 1.5 ms or lower) even considering the common configurations varying independently the number of neurons per hidden layer (up to 20), the number of hidden layers (up to 20) and the number of training cases (up to 4,000).