Reviewerly/exHarmony huggingface dataset:
IROS 2019 Lifelong Robotic Vision: Object Recognition Challenge [Competitions]

Humans have a remarkable ability to learn continuously from th e environment and inner experience. One of the grand goals of robots is to build an artificial "lifelong learning" agent that can shape a cultivated understanding of the world from the current scene and previous knowledge via an autonomous lifelong development. It is challenging for the robot learning process to retain earlier knowledge when robots encounter new tasks or information. Recent advances in computer vision and deep -learning methods have been impressive due to large-scale data sets, such as ImageNet and COCO. However, robotic vision poses unique new challenges for applying visual algorithms developed from these computer vision data sets because they implicitly assume a fixed set of categories and time -invariant task distributions.
Privacy-Preserving Cost-Sensitive Learning

Cost-sensitive learning methods guaranteeing privacy are becoming crucial nowadays in many applications where increasing use of sensitive personal information is observed. However, there has no optimal learning scheme developed in the literature to learn cost-sensitive classifiers under constraint of enforcing differential privacy. Our approach is to first develop a unified framework for existing cost-sensitive learning methods by incorporating the weight constant and weight functions into the classical regularized empirical risk minimization framework. Then, we propose two privacy-preserving algorithms with output perturbation and objective perturbation methods, respectively, to be integrated with the cost-sensitive learning framework. We showcase how this general framework can be used analytically by deriving the privacy-preserving cost-sensitive extensions of logistic regression and support vector machine. Experimental evidence on both synthetic and real data sets verifies that the proposed algorithms can reduce the misclassification cost effectively while satisfying the privacy requirement. A theoretical investigation is also conducted, revealing a very interesting analytic relation, i.e., that the choice of the weight constant and weight functions does not only influence the Fisher-consistent property (population minimizer of expected risk with a specific loss function leads to the Bayes optimal decision rule) but also interacts with privacy-preserving levels to affect the performance of classifiers significantly.
An efficient automated biospeckle indexing strategy using morphological and geo-statistical descriptors

Biospeckle is caused by statistical interference of coherent beam reflected from a surface having temporal variation due to physiological or biochemical activity. For quantitative evaluation of underlying dynamic speckle activity, different point based and full-field indexing based techniques were proposed in the literature. However, most of the existing techniques involve manual region of interest (ROI) selection, and possess considerable variation of index value with different experimental and analysis parameters (viz. number of frames, degree of correlation, specimen heterogeneity, and others). To circumvent these drawbacks, in this work, we proposed an efficient automated biospeckle indexing technique by combining morphological and geo-statistical operators. Performance of the proposed strategy was analyzed and compared in the controlled environment using different modifications of rotating diffuser based simulation model. Robustness of the proposed strategy was also validated experimentally using different bio-specimens (human finger, seed, carrot and gum arabica). Obtained results demonstrated that the proposed technique has high accuracy for all assessed conditions. Multiple object detection capability of morphological operators was also integrated in the proposed technique to assess biospeckle signature of multiple specimens captured in a single stack of frames. Simultaneous dynamicity assessment of multiple objects from a single stack reduced both computational and experimental overheads considerably. The proposed strategy is useful in biospeckle based quality control and automation.
A Machine Learning Based Methodology for Automatic Annotation and Anonymisation of Privacy-Related Items in Textual Documents for Justice Domain

Textual resources annotation is currently performed both manually by human experts selecting hand-crafted features and automatically by any trained systems. Human experts annotations are very accurate but require heavy effort in compilation and most often are not publicly accessible. Automatic approaches save efforts but don't perform yet with the required accuracy, mostly because of the great difficulty and labor required to represent domain experts' knowledge in a machine readable format. This work tackles the issue of automatically annotate plain text resources; it was motivated by the need of supporting Italian justice officers in detecting sensible information included in large amounts of judgements documents, for privacy preservation aims. We suggest a novel methodology, based on unsupervised machine learning techniques, to facilitate human experts in detecting sensible information. We performed experiments over about 20.000 plain text documents and we obtained an accuracy rate of about 75%, in the preliminary validation stage.
Search Result Explanations Improve Efficiency and Trust

Search engines often provide only limited explanation on why results are ranked in a particular order. This lack of transparency prevents users from understanding results and can potentially give rise to biased or unfair systems. Opaque search engines may also hurt user trust in the presented ranking. This paper presents an investigation of system quality when different degrees of explanation are provided on search engine result pages. Our user study demonstrates that the inclusion of even simplistic explanations leads to better transparency, increased user trust and better search efficiency.
Image Denoising Using DnCNN: An Exploration Study

Image denoising is a crucial pre-processing step on images to restore the original image by suppressing the associated noise. This paper extends the performance study of the denoising convolutional neural network (DnCNN) architecture on images having the Gaussian noise. The DnCNN is an efficient deep learning model to estimate a residual image from the input image with the Gaussian noise. The underlying noise-free image can be estimated as the difference between the noisy image and the residue image. In this paper, we analyse the performance of DnCNN with data augmentation, batch normalisation and dropout. The experiments are conducted on the Berkeley natural image dataset, and quantitative and qualitative study has been performed. The comparison of the experimental results demonstrates that the DnCNN model converges at a faster rate and works well with a smaller dataset.
StackNet: Stacking feature maps for Continual learning

Training a neural network for a classification task typically assumes that the data to train are given from the beginning. However, in the real world, additional data accumulate gradually and the model requires additional training without accessing the old training data. This usually leads to the catastrophic forgetting problem which is inevitable for the traditional training methodology of neural networks. In this paper, we propose a continual learning method that is able to learn additional tasks while retaining the performance of previously learned tasks by stacking parameters. Composed of two complementary components, the index module and the StackNet, our method estimates the index of the corre¬sponding task for an input sample with the index module and utilizes a particular portion of StackNet with this index. The StackNet guarantees no degradation in the performance of the previously learned tasks and the index module shows high confidence in finding the origin of an input sample. Compared to the previous work of PackNet, our method is competitive and highly intuitive.
KLOOS: KL Divergence-based Out-of-Scope Intent Detection in Human-to-Machine Conversations

User intent is not restricted in human-to-machine conversations, and sometimes overshoots the scope of a designed system. Many tasks for understanding conversations require the elimination of such out-of-scope queries. We propose an out-of-scope intent detection method, called KLOOS, based on a novel feature extraction mechanism that incorporates the information accumulation of sequential word processing. Information is accumulated by KL divergence between the intent distributions of consecutive words. The performance of our approach is compared with the conventional classifiers and state-of-the-art language models fine-tuned for out-of-scope detection on three spoken query collections. The results show that KLOOS statistically significantly improves out-of-scope sensitivity in all cases, while the overall performance is not deteriorated in most cases.
Improving RGB-D Face Recognition via Transfer Learning from a Pretrained 2D Network

2D Face recognition has been extensively studied for decades and has reached remarkable results in recent years. However, 2D Face recognition is sensitive to variations in poses, facial expressions and illuminations. Depth images provide valuable information to help model facial boundaries and understand the global facial layout and provide low frequency patterns. Intuitively, RGB-D images are more robust to external environments than RGB images. Unfortunately, RGB-D datasets are orders of magnitude smaller than 2D datasets and insufficient to train a deep CNN model as effective as RGB-based models. To tackle these challenges, we present an RGB-D ResNet50 model which can be transferred from a pretrained RGB model and takes RGB-D images as input. We achieved an accuracy of 94.64% and won the $$1^{\text {st}}$$ place on 3D Face Recognition Algorithm Challenge, 2019 BenchCouncil International Artificial Intelligence System Challenges.
Performance evaluation of optimized and adaptive neuro fuzzy inference system for predictive modeling in agriculture

The Neural Network has a significant impact in developing predictive models in a wide range of applications. In this paper, a neuro-fuzzy prediction model is developed depending on improving the performance of the traditional artificial neural networks using Adaptive Momentum Optimizer. This optimizer simulates the behavior of the International Trade Analysis in the agriculture industry, and this method is used to determine the optimal parameters of artificial neural networks. The proposed model is compared with the existing models such as Support Vector Machine, Random Forest, Decision Tree and traditional Artificial Neural Network models. To examine the forecasting performance of the proposed approach, agriculture datasets is used. The performance of the models was assessed using different performance evaluation criteria and the empirical results show that the back propagation neural network with Adam optimizer attains favorable prediction accuracy of 96.78%, and a better convergence rate. Compared to other benchmark algorithms, the proposed algorithm performs better, and the result validates the effectiveness of the back propagation with Adam optimizer for Natural Language Processing.
DEVELOPING A COMPRESSION PROCEDURE BASED ON THE WAVELET DENOISING AND JPEG2000 COMPRESSION

Image compression has significant importance due to broad employment of image data in today’s computing and communication systems. In image compression, there is generally a trade-off between image visual quality and the compression rate. Although certain types of applications favor the visual quality of an image, some others can favor the compression rate. Thus, an optimal operating point between these two should be determined. In this study, for this aim, we propose an optimal compression procedure based on transform coding for noisy images. We combine the wavelet-based JPEG2000 compression algorithm with wavelet-based denoising algorithms. Thus, thanks to our procedure, optimum performance can be achieved depending on the image type.
AdaMT-Net: An Adaptive Weight Learning Based Multi-Task Learning Model For Scene Understanding

We tackle the problem of deep end-to-end multi-task learning (MTL) for visual scene understanding from monocular images in this paper. It is proven that learning several related tasks together helps in attaining improved performance per-task than training them independently. This is due to the fact that related tasks share important feature characteristics among themselves, which the MTL techniques can effectively explore for improved joint training. Based on this premise, we are interested in generic to specific feature extraction given the different tasks within a common framework. To this end, we propose a typical U-Net based encoder-decoder architecture called AdaMT-Net, where the densely-connected deep convolutional neural network (CNN) based feature encoder is shared among the tasks while the soft-attention based task-specific decoder modules produce the desired outcomes. One major issue in MTL is to assign the weights for the task-specific loss-terms in the final cumulative optimization function. As opposed to the manual approach, we propose a novel adaptive weight learning strategy by carefully exploring the loss-gradients per-task over the training iterations. Experimental results on the benchmark CityScapes, NYUv2, and ISPRS datasets confirm that AdaMT-Net achieves state-of-the-art performance on most of the evaluation metrics.
Augmented Reality and Machine Learning based Product Identification in Retail using Vuforia and MobileNets

Interactive retail is one of the biggest technology trends in recent years. Machine Learning based object detection and Aug mented Reality have proven as major technological advancements in the retail industry. This proposed research work aims at overcoming several setbacks in the present scenario such as high labor costs, time consuming process of purchasing the products, helping the customers to maintain their specified budgets and language barriers while selling products to groups with different linguistic backgrounds and by combining the real-world interaction of Augmented Reality and Machine Learning-based product identification. The proposed work provides customers with a futuristic shopping experience while maintaining their allowed budgets. The Machine Learning-based product identification approach detected objects with around 96% accuracy and Vuforia-based Augmented Reality approach detected objects with maximum accuracy.
MA360: Multi-Agent Deep Reinforcement Learning Based Live 360-Degree Video Streaming on Edge

The mobile edge caching has made video service providers deliver live 360-degree videos worldwide. However, these services still suffer from the huge network traffic on the core network due to the spherical nature and the diverse requests generated from large user populations. It is challenging to optimize the Quality of Experience (QoE) and the bandwidth consumption simultaneously under the significant number of users as well as dynamic network and playback status. In this paper, we propose a Multi-Agent deep reinforcement learning based 360-degree video streaming system, named MA360, to tackle this multi-user live 360-degree video streaming problem in the context of the edge cache network. Specifically, MA360 employs the Mean Field Actor-Critic (MFAC) algorithm to make clients collaboratively and distributively request tiles aiming at maximizing the overall QoE while minimizing the total bandwidth consumption. Experiments over real-world datasets show that MA360 can improve the QoE while significantly reducing the bandwidth consumption compared with several state-of-the-art edge-assisted 360-degree video streaming strategies.
Relative Interior Rule in Block-Coordinate Descent

It is well-known that for general convex optimization problems, block-coordinate descent can get stuck in poor local optima. Despite that, versions of this method known as convergent message passing are very successful to approximately solve the dual LP relaxation of the MAP inference problem in graphical models. In attempt to identify the reason why these methods often achieve good local minima, we argue that if in block-coordinate descent the set of minimizers over a variable block has multiple elements, one should choose an element from the relative interior of this set. We show that this rule is not worse than any other rule for choosing block-minimizers. Based on this observation, we develop a theoretical framework for block-coordinate descent applied to general convex problems. We illustrate this theory on convergent message-passing methods.
PatchVAE: Learning Local Latent Codes for Recognition

Unsupervised representation learning holds the promise of exploiting large amounts of unlabeled data to learn general representations. A promising technique for unsupervised learning is the framework of Variational Auto-encoders (VAEs). However, unsupervised representations learned by VAEs are significantly outperformed by those learned by supervised learning for recognition. Our hypothesis is that to learn useful representations for recognition the model needs to be encouraged to learn about repeating and consistent patterns in data. Drawing inspiration from the mid-level representation discovery work, we propose PatchVAE, that reasons about images at patch level. Our key contribution is a bottleneck formulation that encourages mid-level style representations in the VAE framework. Our experiments demonstrate that representations learned by our method perform much better on the recognition tasks compared to those learned by vanilla VAEs.
Mobileportation

The desire to stay connected to one another over large distances has guided decades of telepresence research. Most focuses on stationary solutions that can deliver high-fidelity telepresence experiences, but these are usually impractical for the wider population who cannot afford the necessary proprietary equipment or are unwilling to regress to non-mobile communication. In this paper we present Mobileportation, a nomadic telepresence prototype that takes advantage of recent developments in mobile technology to provide immersive experiences wherever the user desires by allowing for seamless transitions between ego- and exocentric views within a mutually shared three-dimensional environment. The results of a user study are also discussed that show Mobileportation's ability to induce a sense of presence within this environment and with the remote communication partner, as well as the potential of this platform for future telepresence research.
Stochastic Gradient and Langevin Processes

We prove quantitative convergence rates at which discrete Langevin-like processes converge to the invariant distribution of a related stochastic differential equation. We study the setup where the additive noise can be non-Gaussian and state-dependent and the potential function can be non-convex. We show that the key properties of these processes depend on the potential function and the second moment of the additive noise. We apply our theoretical findings to studying the convergence of Stochastic Gradient Descent (SGD) for non-convex problems and corroborate them with experiments using SGD to train deep neural networks on the CIFAR-10 dataset.
Consensus Learning for Distributed Fuzzy Neural Network in Big Data Environment

Uncertainty and distributed nature inherently exist in big data environment.Distributed fuzzy neural network (D-FNN) that not only employs fuzzy logics to alleviate the uncertainty problem but also deal with data in a distributed manner, is effective and crucial for big data.Existing D-FNNs always avoided consensus for their antecedent layer due to computational difficulty.Hence such D-FNNs are not really distributed since a single model can not be agreed by multiple agents.This paper proposes a real D-FNN model to handle the uncertainty and distributed challenges in the big data environment.The proposed D-FNN model considers consensus for both the antecedent and consequent layers.A novel consensus learning, which involves a distributed structure learning and a distributed parameter learning, is proposed to handle the D-FNN model.The proposed consensus learning algorithm is built on the well-known alternating direction method of multipliers, which does not exchange local data among agents.The major contribution of this paper is to propose the real D-FNN model for the big data and the novel consensus learning algorithm for this D-FNN model.Simulation results on widespread datasets demonstrate the superiority and effectiveness of the proposed D-FNN model and consensus learning algorithm.
Survey on Learning-Based Formal Methods: Taxonomy, Applications and Possible Future Directions

Formal methods play an important role in testing and verifying software quality, especially in modern society with rapid technological updates.Learning-based techniques have been extensively applied to learn (a model or model-free) for formal verification and to learn system specifications, and resulted in numerous contributions.Due to the fact that adequate system models are often difficult to design manually and manual definition of specifications for such software systems gets infeasible, which motivate new research directions in learning models and/or specifications from observed system behaviors automatically.This paper mainly concentrates on learning-based techniques in formal methods area.An upto-date overview of the current state-of-the-art in learning-based formal methods is provided in the paper.This paper is not a comprehensive survey of learning-based techniques in formal methods area, but rather as a survey of the taxonomy, applications and possible future directions in learning-based formal methods.
CLEval: Character-Level Evaluation for Text Detection and Recognition Tasks

Despite the recent success of text detection and recognition methods, existing evaluation metrics fail to provide a fair and reliable comparison among those methods. In addition, there exists no end-to-end evaluation metric that takes characteristics of OCR tasks into account. Previous end-to-end metric contains cascaded errors from the binary scoring process applied in both detection and recognition tasks. Ignoring partially correct results raises a gap between quantitative and qualitative analysis, and prevents fine-grained assessment. Based on the fact that character is a key element of text, we hereby propose a Character-Level Evaluation metric (CLEval). In CLEval, the instance matching process handles split and merge detection cases, and the scoring process conducts character-level evaluation. By aggregating character-level scores, the CLEval metric provides a fine-grained evaluation of end-to-end results composed of the detection and recognition as well as individual evaluations for each module from the end-performance perspective. We believe that our metrics can play a key role in developing and analyzing state-of-the-art text detection and recognition methods. The evaluation code is publicly available at https://github.com/clovaai/CLEval.