Reviewerly/exHarmony huggingface dataset:
STAR-Caps: Capsule Networks with Straight-Through Attentive Routing

Capsule networks have been shown to be powerful models for image classification, thanks to their ability to represent and capture viewpoint variations of an object. However, the high computational complexity of capsule networks that stems from the recurrent dynamic routing poses a major drawback making their use for large-scale image classification challenging. In this work, we propose Star-Caps a capsule-based network that exploits a straight-through attentive routing to address the drawbacks of capsule networks. By utilizing attention modules augmented by differentiable binary routers, the proposed mechanism estimates the routing coefficients between capsules without recurrence, as opposed to prior related work. Subsequently, the routers utilize straight-through estimators to make binary decisions to either connect or disconnect the route between capsules, allowing stable and faster performance. The experiments conducted on several image classification datasets, including MNIST, SmallNorb, CIFAR-10, CIFAR-100, and ImageNet show that Star-Caps outperforms the baseline capsule networks.
Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

Recent work has shown that CNN-based depth and ego-motion estimators can be learned using unlabelled monocular videos. However, the performance is limited by unidentified moving objects that violate the underlying static scene assumption in geometric image reconstruction. More significantly, due to lack of proper constraints, networks output scale-inconsistent results over different samples, i.e., the ego-motion network cannot provide full camera trajectories over a long video sequence because of the per-frame scale ambiguity. This paper tackles these challenges by proposing a geometry consistency loss for scale-consistent predictions and an induced self-discovered mask for handling moving objects and occlusions. Since we do not leverage multi-task learning like recent works, our framework is much simpler and more efficient. Comprehensive evaluation results demonstrate that our depth estimator achieves the state-of-the-art performance on the KITTI dataset. Moreover, we show that our ego-motion network is able to predict a globally scale-consistent camera trajectory for long video sequences, and the resulting visual odometry accuracy is competitive with the recent model that is trained using stereo videos. To the best of our knowledge, this is the first work to show that deep networks trained using unlabelled monocular videos can predict globally scale-consistent camera trajectories over a long video sequence.
End to end learning and optimization on graphs

Real-world applications often combine learning and optimization problems on graphs. For instance, our objective may be to cluster the graph in order to detect meaningful communities (or solve other common graph optimization problems such as facility location, maxcut, and so on). However, graphs or related attributes are often only partially observed, introducing learning problems such as link prediction which must be solved prior to optimization. Standard approaches treat learning and optimization entirely separately, while recent machine learning work aims to predict the optimal solution directly from the inputs. Here, we propose an alternative decision-focused learning approach that integrates a differentiable proxy for common graph optimization problems as a layer in learned systems. The main idea is to learn a representation that maps the original optimization problem onto a simpler proxy problem that can be efficiently differentiated through. Experimental results show that our ClusterNet system outperforms both pure end-to-end approaches (that directly predict the optimal solution) and standard approaches that entirely separate learning and optimization. Code for our system is available at https://github.com/bwilder0/clusternet.
Detecting Adversarial Perturbations Through Spatial Behavior in Activation Spaces

Although neural network-based classifiers outperform humans in a range of tasks, they are still prone to manipulation through adversarial perturbations. Prior research has resulted in the identification of effective defense mechanisms for many reported attack methods, however a defense against the C&W attack, as well as a holistic defense mechanism capable of countering multiple different attack methods, are still missing. All attack methods reported so far share a common goal. They aim to avoid detection by limiting the allowed perturbation magnitude, and still trigger incorrect classification. As a result, small perturbations cause classification to shift from one class to another. We coined the term activation spaces to refer to the hyperspaces formed by the activation values of the different network layers. We then use activation spaces to capture the differences in spatial dynamics between normal and adversarial examples, and form a novel adversarial example detector. We induce a set of k-nearest neighbor (k-NN) classifiers, one per activation space, and leverage those classifiers to assign a sequence of class labels to each input of the neural network. We then calculate the likelihood of each observed label sequence and show that sequences associated with adversarial examples are far less likely than those of normal examples. We demonstrate the efficiency of our proposed detector against the C&W attack using two image classification datasets (MNIST, CIFAR-10) achieving an AUC of 0.97 for the CIFAR-10 dataset. We further show how our detector can be easily augmented with previously suggested defense methods to form a holistic multi-purpose defense mechanism.
A fully automatic segmentation algorithm for CT lung images based on random forest

Several negative factors, such as juxta-pleural nodules, pulmonary vessels, and image noise, make accurately segmenting lungs from computed tomography (CT) images a complex task. We propose a novel hybrid automated algorithm in the paper based on random forest to deal with the issues. Our method aims to eliminate the effect of the factors and generate accurate segmentation of lungs from CT images.Our algorithm consists of five main steps: image preprocessing, lung region extraction, trachea elimination, lung separation, and contour correction. A lung CT image is first preprocessed with a novel normal vector correlation-based image denoising approach and decomposed into a group of multiscale subimages. A modified superpixel segmentation method is then performed on the first-level subimage to generate a set of superpixels, and a random forest classifier is employed to segment the lungs by classifying the superpixels of each subimage-based on the features extracted from them. The initial lung segmentation result is further refined through trachea elimination using an iterative thresholding approach, lung separation based on context information of image sequence, and contour correction with a corner detection technique.Our algorithm is tested on a set of CT images affected with interstitial lung diseases, and experiments show that the algorithm achieves high accuracy on lung segmentation with 0.9638 Jaccard's index and 0.9867 Dice similarity coefficient, compared with ground truths. Additionally, our algorithm achieves an average 7.7% better Dice similarity coefficient than compared conventional lung segmentation methods and 1% better than Deep Learning.Our algorithm can segment lungs from lung CT images with good performance in a fully automatic fashion, and it is of great assistance for lung disease detection in the computer-aided detection system.
Abstractive summarization of long texts by representing multiple compositionalities with temporal hierarchical pointer generator network

In order to tackle the problem of abstractive summarization of long multi-sentence texts, it is critical to construct an efficient model, which can learn and represent multiple compositionalities better. In this paper, we introduce a temporal hierarchical pointer generator network that can represent multiple compositionalities in order to handle longer sequences of texts with a deep structure. We demonstrate how a multilayer gated recurrent neural network organizes itself with the help of an adaptive timescale in order to represent the compositions. The temporal hierarchical network is implemented with a multiple timescale architecture where the timescale of each layer is also learned during the training process through error backpropagation through time. We evaluate our proposed model using an Introduction-Abstract summarization dataset from scientific articles and the CNN/Daily Mail summarization benchmark dataset. The results illustrate that, we successfully implement a summary generation system for long texts by using the multiple timescale with adaptation concept. We also show that we have improved the summary generation system with our proposed model on the benchmark dataset.
Semantic Consistency Cross-Modal Retrieval With Semi-Supervised Graph Regularization

Most of the existing cross-modal retrieval methods make use of labeled data to learn projection matrices for different modal data. These methods usually learn the original semantic space to bridge the heterogeneous gap, ignoring the rich semantic information contained in unlabeled data. Accordingly, a semantic consistency cross-modal retrieval with semi-supervised graph regularization (SCCMR) algorithm is proposed, which integrates the prediction of labels and the optimization of projection matrices into a unified framework to ensure that the solution obtained is globally optimal. At the same time, the method uses graph embedding to consider the nearest neighbors in the potential subspace of paired images and texts as well as images and texts with the same semantics. l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">21</sub> -norm constraint is applied to the projection matrices to select the discriminative features for different modal data. The results show that our method outperforms several advanced methods on four commonly used cross-modal retrieval datasets.
SalsaNext: Fast Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving.

In this paper, we introduce SalsaNext for the uncertainty-aware semantic segmentation of a full 3D LiDAR point cloud in real-time. SalsaNext is the next version of SalsaNet [1] which has an encoder-decoder architecture where the encoder unit has a set of ResNet blocks and the decoder part combines upsampled features from the residual blocks. In contrast to SalsaNet, we introduce a new context module, replace the ResNet encoder blocks with a new residual dilated convolution stack with gradually increasing receptive fields and add the pixel-shuffle layer in the decoder. Additionally, we switch from stride convolution to average pooling and also apply central dropout treatment. To directly optimize the Jaccard index, we further combine the weighted cross-entropy loss with Lovasz-Softmax loss [2]. We finally inject a Bayesian treatment to compute the epistemic and aleatoric uncertainties for each point in the cloud. We provide a thorough quantitative evaluation on the Semantic-KITTI dataset [3], which demonstrates that the proposed SalsaNext outperforms other state-of-the-art semantic segmentation networks and ranks first on the Semantic-KITTI leaderboard. We also release our source code this https URL.
A Comparative Evaluation of Texture Features for Semantic Segmentation of Breast Histopathological Images

Breast histopathological image analysis helps in understanding the structure and distribution of the nucleus, thereby assisting in the detection of breast cancer. But analysis of histopathological image is challenging due to various reasons such as heterogeneity of nucleus structure, overlapping nuclei, clustered nuclei, variations in illumination, presence of noise etc. Limited availability of breast histopathological image dataset with fine annotations for detection of nucleus has restricted the analysis of histopathological images at the pixel-level. This paper presents fine annotations for nucleus segmentation of breast histopathological image datasets. Various textures such as Filter Banks, Gray Level Co-occurrence matrix and Local Binary Patterns are studied along with colour features for semantic segmentation of nuclei from histopathological images. Support Vector Machine and Multi Layer Perceptron algorithms are trained to perform pixelwise classification. The performance of the three texture features are evaluated on the two datasets and the results are presented in this paper.
Adaptive Switching Spatial-Temporal Fusion Detection for Remote Flying Drones

The drone has been applied in various areas due to its small size, high mobility and low price. However, illegal uses of drones have posed huge threats to both public safety and personal privacy. There is an urgent demand for technologies that can timely detect and counter the drones. In this paper, we propose an adaptive switching spatial-temporal fusion detection method for remote flying drones in the airspace using electrical-optical cameras, which can enhance the contrast between the target and background as well as suppressing the noises and clutters simultaneously. For each incoming video frame, a dark-attentive interframe difference method and a row-column separate black-hat method are proposed to generate temporal feature maps (TFM) and spatial feature maps (SFM), respectively, in parallel. Inspired by the phenomenon that the features in TFMs and SFMs both go strong at the regions of the intended target while they do not at other regions where noises and clutters locate, we design an adaptive switching spatial-temporal fusion mechanism to fuse the SFMs and TFMs, generating adaptive switching spatial-temporal feature maps (ASSTFM). Finally, an adaptive local threshold mechanism is used in ASSTFMs to segment the targets from backgrounds. In order to validate the effectiveness of our method, we conduct both offline experiments and field tests. The experiment results manifest that our method is superior to the other seven baseline methods and works more stably for different backgrounds and various types of drones.
MC-SVM Based Work Flow Preparation in Cloud with Named Entity Identification

The existing applications that are associated with the internet produce enormous amount of data according to the requirements of diverse circumstances prevailing. This causes multitudes of challenges in examining the data and as well as in the operation of the system that relies on the cloud. To simply process and manage the execution of the tasks properly with respect to time the workflow scheduling was devised in the cloud. To further enhance the process of scheduling the named entity recognition is used. The NER-named entity recognition is an important chore of more general discipline of internet explorer application. Since the NER- problem is highly challenging in cloud paradigm. An innovative frame work termed as the MC-SVM (Multi Class- Support Vector Machine) is laid out in the paper to devise the scheduling of the workflow in the cloud paradigm. The scheduling of the tasks in the cloud delivers an arrangement setting up the work flows with the named entity recognition using the MC-SVM. The algorithm developed enhances the resource allocation process, by performing a simultaneous and dynamic allocation/reallocation of named entities to the resources of the cloud satisfying the demands in the performance and cost. The results observed on validating the proposed algorithm proves the capability of the system to manage the resources in the cloud effectively optimizing the make span and the cost.
Deep Autoencoder Learning for Relay-Assisted Cooperative Communication Systems

Emerging recently as a novel concept in communication system design, end-to-end learning introduces deep neural networks (NNs) to represent the transmitter and receiver functions. Consequently, the whole system can be interpreted as an autoencoder (AE), which can be optimized from a holistic approach through a data-driven training method. Until now, the AE technique is mainly developed for point-to-point communication scenarios. In this paper, we aim to develop a novel NN-based AE scheme for relay-assisted cooperative communication systems. Specifically, three NN components are constructed to learn the behavior of the transmitter, relay node, and receiver, respectively. As the conventional end-to-end training is inapplicable, a novel two-stage training approach is proposed to indirectly solve the end-to-end training problem. The implicit approximations involved are analytically expressed based on information theory, offering insights on the achievable performance with the proposed training method. The proposed AE model eliminates the need for channel state information and noise variance of any link, and is adaptive to the variation in the input block length. Simulation results verify its advantages over the conventional decode-and-forward (DF) and amplify-and-forward (AF) schemes in various scenarios.
Online Learned Continual Compression with Adaptive Quantization Modules

We introduce and study the problem of Online Continual Compression, where one attempts to simultaneously learn to compress and store a representative dataset from a non i.i.d data stream, while only observing each sample once. A naive application of auto-encoders in this setting encounters a major challenge: representations derived from earlier encoder states must be usable by later decoder states. We show how to use discrete auto-encoders to effectively address this challenge and introduce Adaptive Quantization Modules (AQM) to control variation in the compression ability of the module at any given stage of learning. This enables selecting an appropriate compression for incoming samples, while taking into account overall memory constraints and current progress of the learned compression. Unlike previous methods, our approach does not require any pretraining, even on challenging datasets. We show that using AQM to replace standard episodic memory in continual learning settings leads to significant gains on continual learning benchmarks. Furthermore we demonstrate this approach with larger images, LiDAR, and reinforcement learning environments.
A Risk-based UAS Traffic Network Model for Adaptive Urban Airspace Management

This paper presents a risk-based model for UAV path planning in urban low altitude environments.Firstly, the risks in urban environments were identified and classified into four groups, and the corresponding risk costs were categorized into five different levels.A general risk cost model was then developed for qualifying all related risk costs.Secondly, a ConOps for urban air route network with plane and cubical diagonals was built and the obtained risk cost information were ingested into the network to assist path planning.Finally, an A*cost algorithm was developed to generate a safe and cost-effective path for UAVs in urban low altitude airspace.Simulation results show that the novel route network structure with diagonals has better performance, with an average reduction of 17.56% for path distance.While the proposed A*cost algorithm has good results in terms of total cost and computational time, outperforming traditional Dijkstra and Ant Colony algorithm.
Multimedia English teaching analysis based on deep learning speech enhancement algorithm and robust expression positioning

In multimedia English teaching, learners face such an indifferent computer screen without emotion and feel the fun of interaction and emotional stimulation, which will cause resentment and affect the learner’s learning effect. In order to improve the efficiency of multimedia English teaching, aiming at the lack of emotion in multimedia English education, this study proposes an intelligent network teaching system model based on deep learning speech enhancement and facial expression recognition. Moreover, this study uses emotional calculation as the theoretical basis and uses facial expression recognition as the core technology to judge and understand the emotional state by capturing and recognizing the facial expressions of online learners. In addition, this study has carried out experimental tests on the effect of the identification method of this paper and verified that the method has good detection effect on the real smile micro-expressions through two sets of experiments and can provide theoretical reference for subsequent related research.
A hybrid constriction coefficient-based particle swarm optimization and gravitational search algorithm for training multi-layer perceptron

Purpose In this paper, a newly proposed hybridization algorithm namely constriction coefficient-based particle swarm optimization and gravitational search algorithm (CPSOGSA) has been employed for training MLP to overcome sensitivity to initialization, premature convergence, and stagnation in local optima problems of MLP. Design/methodology/approach In this study, the exploration of the search space is carried out by gravitational search algorithm (GSA) and optimization of candidate solutions, i.e. exploitation is performed by particle swarm optimization (PSO). For training the multi-layer perceptron (MLP), CPSOGSA uses sigmoid fitness function for finding the proper combination of connection weights and neural biases to minimize the error. Secondly, a matrix encoding strategy is utilized for providing one to one correspondence between weights and biases of MLP and agents of CPSOGSA. Findings The experimental findings convey that CPSOGSA is a better MLP trainer as compared to other stochastic algorithms because it provides superior results in terms of resolving stagnation in local optima and convergence speed problems. Besides, it gives the best results for breast cancer, heart, sine function and sigmoid function datasets as compared to other participating algorithms. Moreover, CPSOGSA also provides very competitive results for other datasets. Originality/value The CPSOGSA performed effectively in overcoming stagnation in local optima problem and increasing the overall convergence speed of MLP. Basically, CPSOGSA is a hybrid optimization algorithm which has powerful characteristics of global exploration capability and high local exploitation power. In the research literature, a little work is available where CPSO and GSA have been utilized for training MLP. The only related research paper was given by Mirjalili et al. , in 2012. They have used standard PSO and GSA for training simple FNNs. However, the work employed only three datasets and used the MSE performance metric for evaluating the efficiency of the algorithms. In this paper, eight different standard datasets and five performance metrics have been utilized for investigating the efficiency of CPSOGSA in training MLPs. In addition, a non-parametric pair-wise statistical test namely the Wilcoxon rank-sum test has been carried out at a 5% significance level to statistically validate the simulation results. Besides, eight state-of-the-art meta-heuristic algorithms were employed for comparative analysis of the experimental results to further raise the authenticity of the experimental setup.