Reviewerly/exHarmony huggingface dataset:
Random mask-based estimation of the distribution algorithm for stacked auto-encoder one-step pre-training

The deep learning techniques have received great achievements in computer vision, natural language processing, etc. The success of deep neural networks depends on the sufficient training of parameters. The traditional way of neural network training is a gradient-based algorithm, which suffers the disadvantage of gradient disappearing, especially for the deeper neural network. Recently, a heuristic algorithm has been proposed for deeper neural network optimization. In this paper, a random mask and elitism univariate continuous estimation of distribution algorithm based on the Gaussian model is proposed to pre-train staked auto-encoder, and then a Stochastic Gradient Descent (SGD) based fine-tuning process is carried out for local searching. In the improved estimation of the distribution algorithm, two individual update strategies are defined; one group of individuals is generated according to the constructed probabilistic model, and another is updated according to the statistics of advanced individuals that aim to reduce the probability of combination explosion and time consumption according to the mask information. In the simulations, different architectures, different mask ratios and different promising individual ratios are adopted to testify the effectiveness of the improved algorithm. According to simulation results, the estimation of thr distribution algorithm has a steady optimization ability for the shallow and stacked auto-encoder by one-step pre-training combining SGD based fine-tuning for the MNIST dataset. The proposed model will achieve a state-of-the-art performance on Fashion-MNIST.
Temporal Feature Enhancement Network with External Memory for Object Detection in Surveillance Video

Video object detection is challenging and essential in practical applications, such as surveillance cameras for traffic control and public security. Unlike the video in natural scenes, the surveillance video tends to contain dense and small objects (typically vehicles) in their appearances. Therefore, existing methods for surveillance object detection utilize still-image object detection approaches with rich feature extractors at the expense of their run-time speeds. The run-time speed, however, becomes essential when the video is being streamed. In this paper, we exploit temporal information in videos to enrich the feature maps, proposing the first temporal attention based external memory network for the live stream of video. Extensive experiments on real-world traffic surveillance benchmarks demonstrate the real-time performance of the proposed model while keeping comparable accuracy with state-of-the-art.
Improving the search pattern of Rooted Tree Optimisation algorithm to solve complex problems

Rooted Tree Optimisation (RTO) is a metaheuristic method for solving complex problems. Roots in RTO move to new search space without evaluating such move for better fitness values. Best root in RTO which acts similar to the global best in Particle Swarm Optimisation, is used to influence the movement of the rest of the roots to a promising region. This influence may lead the algorithm to premature convergence. The Lateral Growth Rooted Tree Optimisation (LGRTO) that is proposed in this work eliminates the influence of the best root to direct the search pattern of other roots. This is achieved by introducing hydrotropism and lateral growth equations. In both equations, the influence of the best root is eliminated, and every exploitation or exploration is evaluated before the roots move to new points in the search space. Three types of experiments were done. Two of these experiments are benchmark functions for continuous optimisation problems. The third type is a fourth order Butterworth filter design problem. The result of the nonparametric test of experiments on continuous optimisation problem indicated that LGRTO obtained better performance over Rooted Tree Optimisation and Real Coded Genetic Algorithm. The performance of LGRTO is competitive with the result of Teaching and Learning Based Optimisation (TLBO) and LSHADE with Semi-Parameter Adaptation Hybrid with CMA-ES (LSHADE-SPACMA) methods. For the result of Butterworth design problem, LGRTO obtained a design error that is lower than the value obtained by RTO, Particle Swarm Optimisation (PSO), Differential Evolution (DE), Tabu Search (TS), and Artificial Bee Colony (ABC) methods.
ORP: An Online Rule Placement Scheme to Optimize the Traffic Overhead for Data Center Networks

In cloud data centers, the requirement for fine-grained policy leads to a large number of server rules. However, the servers cannot efficiently handle all of these rules due to limitations of the central processing unit (CPU) and memory resources. Thus, a rule partition and placement algorithms have been proposed to divide the original ruleset into several rule subsets and migrate some rule subsets to other devices. Correspondingly, the traffic, which is meant to be blocked by the original server's migratory rules, will also be redirected to the new device. The rule partition and placement algorithms should minimize this extra traffic overhead. The state-of-the-art algorithm, vCRIB, is offline, as its solution is calculated based on a static traffic matrix. In reality, when facing dynamic traffic, the vCRIB solution may actually be suboptimal in terms of traffic overhead. To reduce the traffic overhead, we propose the online rule replacement (ORP) algorithm. Specifically, ORP utilizes the difference between the old and new traffic matrices to discover the rule subsets matching the notably changed traffic overhead or blocked traffic. These rule subsets are then preferentially adjusted to their proper placement. In this manner, ORP can promptly make rule subset adjustments according to the newly updated traffic matrix. Additionally, ORP effectively avoids the invalid migrations existing in vCRIB. The simulation results confirm that ORP is capable of reducing traffic overhead by 26.67%–56.9% compared to vCRIB.
Fractional Adaptation of Activation Functions In Neural Networks

In this work, we introduce a generalization methodology for the automatic selection of the activation functions inside a neural network, taking advantage of concepts defined in fractional calculus. This methodology enables the neural network to search and optimize its own activation functions during the training process, by defining the fractional order of the derivative of a given primitive activation function. This fractional order is tuned as an additional training hyper-parameter a for intrafamily selection and b for cross family selection. By following this approach, the neurons inside the network can adjust their activation functions, e.g. from MLP to RBF networks, to best fit the input data, and reduce the output error. The experimental results obtained show the benefits of using this technique implemented on a ResNet18 topology, by outperforming the accuracy of a ResNet100 trained with CIFAR10 and Improving 1% ImageNet reported in the literature.
PLVCG: A Pretraining Based Model for Live Video Comment Generation

Live video comment generating task aims to automatically generate real-time viewer comments on videos like real viewers do. Like providing search suggestions by search engines, this task can help viewers find comments they want to post by providing generated comments. Previous works ignore the interactivity and diversity of comments and can only generate general and popular comments. In this paper, we incorporate post time of the comments to deal with the real-time related comment interactions. We also take the video type labels into consideration to handle the diversity of comments and generate more related and informative comments. To this end, we propose a pre-training based encoder-decoder joint model called PLVCG model. This model is composed of a bidirectional encoder to encode context comments and visual frames jointly as well as an auto-regressive decoder to generate real-time comments and classify the type of the video. We evaluate our model in a large-scale real-world live comment dataset. The experiment results present that our model outperforms the state-of-the-art on live video comment ranking and generating task significantly.
A ten-year literature review of content-based image retrieval (CBIR) studies in the tourism industry

Purpose Due to the worldwide growth of digital image sharing and the maturity of the tourism industry, the vast and growing collections of digital images have become a challenge for those who use and/or manage these image data across tourism settings. To overcome the image indexing task with less labour cost and improve the image retrieval task with less human errors, the content-based image retrieval (CBIR) technique has been investigated for the tourism domain particularly. This paper aims to review the relevant literature in the field to understand these previous works and identify research gaps for future directions. Design/methodology/approach A systematic and comprehensive review of CBIR studies in tourism from the year 2010 to 2019, focussing on journal articles and conference proceedings in reputable online databases, is conducted by taking a comparative approach to critically analyse and address the trends of each fundamental element in these research experiments. Findings Based on the review of the literature, the trends of CBIR studies in tourism is to improve image representation and retrieval by advancing existing feature extraction techniques, contributing novel techniques in the feature extraction process through fine-tuning fusion features and improving image query of CBIR systems. Co-authorship, tourist attraction sector and fusion image features have been in focus. Nonetheless, the number of studies in other tourism sectors and available image databases could be further explored. Originality/value The fact that no existing academic review of CBIR studies in tourism makes this paper a novel contribution.
Encoding feature models using mainstream JSON technologies

Feature modeling is a process for identifying the common and variable parts of a software product line and recording them in a tree-structured feature model. However, feature models can be difficult for mainstream developers to specify and maintain because most tools rely on specialized theories, notations, or technologies. To address this issue, we propose a design that uses mainstream JSON-related technologies to encode and manipulate feature models and then uses the models to generate Web forms for product configuration. This JSON-based design can form part of a comprehensive, interactive environment that enables mainstream developers to specify, store, update, and exchange feature models and use them to configure members of product families.
Plot2API: Recommending Graphic API from Plot via Semantic Parsing Guided Neural Network

Plot-based Graphic API recommendation (Plot2API) is an unstudied but meaningful issue, which has several important applications in the context of software engineering and data visualization, such as the plotting guidance of the beginner, graphic API correlation analysis, and code conversion for plotting. Plot2API is a very challenging task, since each plot is often associated with multiple APIs and the appearances of the graphics drawn by the same API can be extremely varied due to the different settings of the parameters. Additionally, the samples of different APIs also suffer from extremely imbalanced.Considering the lack of technologies in Plot2API, we present a novel deep multi-task learning approach named Semantic Parsing Guided Neural Network (SPGNN) which translates the Plot2API issue as a multi-label image classification and an image semantic parsing tasks for the solution. In SPGNN, the recently advanced Convolutional Neural Network (CNN) named EfficientNet is employed as the backbone network for API recommendation. Meanwhile, a semantic parsing module is complemented to exploit the semantic relevant visual information in feature learning and eliminate the appearance-relevant visual information which may confuse the visual-information-based API recommendation. Moreover, the recent data augmentation technique named random erasing is also applied for alleviating the imbalance of API categories.We collect plots with the graphic APIs used to drawn them from Stack Overflow, and release three new Plot2API datasets corresponding to the graphic APIs of R and Python programming languages for evaluating the effectiveness of Plot2API techniques. Extensive experimental results not only demonstrate the superiority of our method over the recent deep learning baselines but also show the practicability of our method in the recommendation of graphic APIs.
Gait Differences in the Real World and Virtual Reality: The Effect of Prior Virtual Reality Experience

Walking is one of the most influential parts of virtual reality applications, such as training, health care, and education. Prior research has established that people's gait behavior change when walking in virtual reality (VR), as compared to the real world. However, very little research has evaluated how people's gait parameters change over time as they gain more experience with VR, and whether this reduced the difference in gait parameters between the real and virtual worlds. We hypothesize that the more experience people have in VR, the less difference there will be between VR and real-world gait parameters. Collected data out of the experiments, observations, and interviews make mixed-method research involving quantitative and qualitative sections to answer the research question.
Generating Natural Questions from Images for Multimodal Assistants

Generating natural, diverse, and meaningful questions from images is an essential task for multimodal assistants as it confirms whether they have understood the object and scene in the images properly. The research in visual question answering (VQA) and visual question generation (VQG) is a great step. However, this research does not capture questions that a visually-abled person would ask multimodal assistants. Recently published datasets such as KB-VQA, FVQA, and OK-VQA try to collect questions that look for external knowledge which makes them appropriate for multimodal assistants. However, they still contain many obvious and common-sense questions that humans would not usually ask a digital assistant. In this paper, we provide a new benchmark dataset that contains questions generated by human annotators keeping in mind what they would ask multimodal digital assistants. Large scale annotations for thousands of images are expensive and time-consuming, so we also present an effective way of automatically generating questions from unseen images. In this paper, we present an approach for generating diverse and meaningful questions that consider image content and metadata of image (e.g., location, associated keyword). We evaluate our approach using standard evaluation metrics such as BLEU, METEOR, ROUGE, and CIDEr to show the relevance of generated questions with human-provided questions. We also measure the diversity of generated questions using generative strength and inventiveness metrics. We report new state-of-the-art results on the public and our datasets.
The performance of Machine Learning on Low Resolution Image Classifier

The ability of machine learning has become a very famous and important technique for discovering statistically significant patterns in the available data. In this paper, we presented the gradient profile spectral characteristics classification on vertical and horizontal gradient acceleration data, Edge Sketch Image and The Relational Gradient Direction data in low-resolution image input. Various training datasets were learned by CatBoost Classifier to created gradient profile priors. This technique was boosting schemes help to reduce over fitting and improves quality of the model. Due to symmetric tree structure of the CatBoost, it provided fast inference and accelerated the implementation. Several predictive and conventional classification techniques were chosen for performance comparison. The experimental results demonstrated performance improvement in classification of the frequency level area in various image characteristics.
Computational tools for inversion and uncertainty estimation in respirometry

In many physiological systems, real-time endogeneous and exogenous signals in living organisms provide critical information and interpretations of physiological functions; however, these signals or variables of interest are not directly accessible and must be estimated from noisy, measured signals. In this paper, we study an inverse problem of recovering gas exchange signals of animals placed in a flow-through respirometry chamber from measured gas concentrations. For large-scale experiments (e.g., long scans with high sampling rate) that have many uncertainties (e.g., noise in the observations or an unknown impulse response function), this is a computationally challenging inverse problem. We first describe various computational tools that can be used for respirometry reconstruction and uncertainty quantification when the impulse response function is known. Then, we address the more challenging problem where the impulse response function is not known or only partially known. We describe nonlinear optimization methods for reconstruction, where both the unknown model parameters and the unknown signal are reconstructed simultaneously. Numerical experiments show the benefits and potential impacts of these methods in respirometry.
Unidirectional Memory-Self-Attention Transducer for Online Speech Recognition

Self-attention models have been successfully applied in end-to-end speech recognition systems, which greatly improve the performance of recognition accuracy. However, such attention-based models cannot be used in online speech recognition, because these models usually have to utilize a whole acoustic sequences as inputs. A common method is restricting the field of attention sights by a fixed left and right window, which makes the computation costs manageable yet also introduces performance degradation. In this paper, we propose Memory-Self-Attention (MSA), which adds history information into the Restricted-Self-Attention unit. MSA only needs localtime features as inputs, and efficiently models long temporal contexts by attending memory states. Meanwhile, recurrent neural network transducer (RNN-T) has proved to be a great approach for online ASR tasks, because the alignments of RNN-T are local and monotonic. We propose a novel network structure, called Memory-Self-Attention (MSA) Transducer. Both encoder and decoder of the MSA Transducer contain the proposed MSA unit. The experiments demonstrate that our proposed models improve WER results than Restricted-Self-Attention models by 13.5% on WSJ and 7.1% on SWBD datasets relatively, and without much computation costs increase.
Interoperable Blockchain Solution For Digital Identity Management

Identity management is the set of policies, processes, and technologies used for managing access to information systems, safeguarding digital data, and tracking system activities. The existing Identity management system is not that secure, reliable, distributed and trustworthy. All these systems containing digital information are vulnerable to getting hacked or sometimes act as a malicious entity if the data is altered on central servers. Data breaches, including confidential data, are happening regularly. Blockchain on the other hand is one of the most emerging technologies that has gained a lot of attention in the past 5-6 years. Blockchain can address most of the issues with the existing identity management system. Over few years, various dApps have been created using Blockchain Framework (leveraging PKI system) to solve the problem of Digital Identity Management, but at the same time, most of the work done is compromising the feature of decentralization (systems based on Centralization as Verifiable Credentials / Public-Private Keys are stored in the central server for verification). In this system, the authors have proposed an interoperable blockchain solution for Identity Management. The purpose of this research work is to study the feasibility of whether or not we can use interoperable blockchain solutions for Identity and Access Management. The authors were able to create a PoC for illustrating the same. This system was able to complete the authentication and verification process within 1 second, and all the necessary functions created to test the feasibility was able to execute within 1 second. Also in this system authors have achieved full decentralization.
Development of a person’s facial expression recognition system using a convolutional neural network

Abstract Humans naturally and intuitively use important and powerful facial expressions to communicate and show their emotions during social interactions. In that interaction, the effort to interpret someone’s emotional condition is important in good communication. A person’s emotional condition is reflected in the form of words, gestures, and especially facial expressions. Although humans can recognize expressions well, research on expression recognition carried out by machines is continuously being carried out in order to be able to carry out expression recognition in human and computer interactions. This research conducted facial expression recognition using the Convolutional Neural Network method. To perform facial expression recognition, the Convolutional Neural Network is trained with expression image data. The training process is carried out using different optimizer values, batch size, and epoch to get the best model. To overcome overfitting, data augmentation was carried out on training data and validation data. The experimental results in this research indicate that the system method built is able to recognize a person’s facial expressions with an accuracy rate of 80%.
Hierarchical Learning of Dependent Concepts for Human Activity Recognition

In multi-class classification tasks, like human activity recognition, it is often assumed that classes are separable. In real applications, this assumption becomes strong and generates inconsistencies. Besides, the most commonly used approach is to learn classes one-by-one against the others. This computational simplification principle introduces strong inductive biases on the learned theories. In fact, the natural connections among some classes, and not others, deserve to be taken into account. In this paper, we show that the organization of overlapping classes (multiple inheritances) into hierarchies considerably improves classification performances. This is particularly true in the case of activity recognition tasks featured in the SHL dataset. After theoretically showing the exponential complexity of possible class hierarchies, we propose an approach based on transfer affinity among the classes to determine an optimal hierarchy for the learning process. Extensive experiments show improved performances and a reduction in the number of examples needed to learn.
A Decentralized Variance-Reduced Method for Stochastic Optimization Over Directed Graphs

In this paper, we propose a decentralized first-order stochastic optimization method Push-SAGA for finite-sum minimization over a strongly connected directed graph. This method features local variance reduction to remove the uncertainty caused by random sampling of the local gradients, global gradient tracking to address the distributed nature of the data, and push-sum consensus to handle the imbalance caused by the directed nature of the underlying graph. We show that, for a sufficiently small step-size, Push-SAGA linearly converges to the optimal solution for smooth and strongly convex problems, making it the first linearly-convergent stochastic algorithm over arbitrary strongly-connected directed graphs. We illustrate the behavior and convergence properties of Push-SAGA with the help of numerical experiments for strongly convex and non-convex problems.