Reviewerly/exHarmony huggingface dataset:
Nighttime Pedestrian Detection Based on Thermal Imaging and Convolutional Neural Networks

Pedestrian detection is a high-profile topic in computer vision, in part because it has great relevance to autonomous driving and intelligent surveillance applications.However, most pedestrian detection algorithms perform stably only during the daytime with sufficient illumination.At night, there is still room for improvement and many challenges exist.These challenges include occlusion caused by objects or crowds, and the problem of image background segmentation caused by environments with varying illumination.In this paper, we propose a nighttime thermal image pedestrian detection system, which can be viewed as an extension of the Faster region-based convolutional neural network (R-CNN) method.The proposed system can be used for static surveillance scenarios.First, a part model branch is proposed to realize the learning of partial pedestrian block features.Second, a segmentation branch is incorporated to strengthen the positioning of the pedestrian foreground.Finally, the branches are integrated through the fused loss function to enable joint training and optimization of the detection model.To evaluate the performance of the proposed model, we tested the system with several nighttime surveillance scenes.The experimental results show that the proposed method can effectively deal with the occlusion problem under challenging illumination environments and achieve performance levels superior to those of some state-of-the-art deep-learning pedestrian detection methods.
Intelligent Extraction of a Knowledge Ontology From Global Patents

The growth of global patents increased over the last decade as enterprises and inventors sought greater protection of their intellectual property (IP) rights. Global patents represent state-of-the-art knowledge for given domains. This research develops a hierarchical Latent Dirichlet Allocation (LDA)-based approach as a computational intelligent method to discover topics and form a top-down ontology, a semantic schema, representing the collective patent knowledge. To validate the knowledge extraction, 1,546 smart retailing patents collected from the Derwent Innovation platform from 2011 and 2016 are used to build the domain ontology schema. The patent set focuses on in-use, globally established, and non-disputed IP covering payment, user experience, and information integration for smart retailing. The clustering and LDA-based ontology system automatically build the knowledge map, which identifies the technology trends and the technology gaps enabling the development of competitive R&amp;D and management strategies.
Why Interoperability at Data Level Is Not Sufficient for Enabling pHealth?

Multidisciplinary and highly dynamic pHealth ecosystems according to the 5P Medicine paradigm require careful consideration of systems integration and interoperability within the domains knowledge space. The paper addresses the different aspects or levels of knowledge representation (KR) and management (KM) from cognitive theories (theories of knowledge) and modeling processes through notation up to processing, tooling and implementation. Thereby, it discusses language and grammar challenges and constraints, but also development process aspects and solutions, so demonstrating the limitation of data level considerations. Finally, it presents the ISO 23903 Interoperability and Integration Reference Architecture to solve the addressed problems and to correctly deploy existing standards and work products at any representational level including data models as well as data model integration and interoperability.
Challenges in Evaluating Interactive Visual Machine Learning Systems

In interactive visual machine learning (IVML), humans and machine learning algorithms collaborate to achieve tasks mediated by interactive visual interfaces. This human-in-the-loop approach to machine learning brings forth not only numerous intelligibility, trust, and usability issues, but also many open questions with respect to the evaluation of the IVML system, both as separate components, and as a holistic entity that includes both human and machine intelligence. This article describes the challenges and research gaps identified in an IEEE VIS workshop on the evaluation of IVML systems.
An improvement in the quality of expert finding in community question answering networks

Expert finding in Community Question Answering (CQA) networks such as Stack Overflow is a practical issue facing a challenging problem called vocabulary gap. A widely used approach to overcome this problem is translation model. Different from prior works that only consider the relevancy of translations to a query, we intend to diversify query translations for better coverage of query topics. In this work, we have utilized the idea of clustering to group relevant translations to a given query into different clusters and then select representatives from each cluster as a set of diverse translations. We have proposed two new approaches to cluster translations. In the first one, the Mutual Information was primarily utilized as a similarity measure during clustering. In the second approach, the relevant translations are embedded in a topic space and then clustered in that space. After clustering, we propose two batch and sequential methods to select a diverse set of translations from the resultant clusters. The batch method selects the top most relevant translations from each cluster proportional to the relevancy of that cluster to the user query. The sequential one is an iterative method that looks for the most diverse set of translations considering the previously selected ones. Finally, to rank users, a regression model was utilized to learn how expert and non-expert users differ in using a set of diverse translations in their documents. Experiments on a large dataset generated from Stack Overflow demonstrate that the proposed methods improve the ranking performance over baselines in the expert finding.
Shot based keyframe extraction using edge-LBP approach

Advancement in technology has led to tremendous increase in the online video content that requires efficient and effective content based video analysis approaches. In this regard, efficient approach for abrupt Shot Boundary Detection (SBD) and keyframe extraction has been presented. The proposed method detects abrupt shots by extracting binarized edge information from frames for texture characterisation using Local Binary Pattern (LBP) method. Further, Euclidean distance has been applied on the histogram features constructed and an adaptive threshold is used to detect abrupt shots. During keyframe extraction phase, magnitude gradient using Sobel operator has been extracted from each frame of the segmented shot. Subsequently, magnitude values are transformed into Z-score which describes the position of each pixel in terms of its distance from the mean, when measured in standard deviation units of every frame. Finally, Co-efficient of variation is computed for each frame and the frame possessing the highest value is selected as a keyframe from every shot. Experiments were conducted on TRECVID 2001 dataset to analyze and validate the proposed approach. Experimental result manifest that the proposed SBD and keyframe extraction method outperforms some of the state-of-the-art algorithms with average F1-score of 98.15% and average fidelity measure of 90% respectively.
Self-Expressing Autoencoders for Unsupervised Spoken Term Discovery

Unsupervised spoken term discovery consists of two tasks: finding the acoustic segment boundaries and labeling acoustically similar segments with the same labels. We perform segmentation based on the assumption that the frame feature vectors are more similar within a segment than across the segments. Therefore, for strong segmentation performance, it is crucial that the features represent the phonetic properties of a frame more than other factors of variability. We achieve this via a self-expressing autoencoder framework. It consists of a single encoder and two decoders with shared weights. The encoder projects the input features into a latent representation. One of the decoders tries to reconstruct the input from these latent representations and the other from the self-expressed version of them. We use the obtained features to segment and cluster the speech data. We evaluate the performance of the proposed method in the Zero Resource 2020 challenge unit discovery task. The proposed system consistently outperforms the baseline, demonstrating the usefulness of the method in learning representations.
Multistep, automatic and nonrigid image registration method for histology samples acquired using multiple stains

The use of multiple dyes during histological sample preparation can reveal distinct tissue properties. However, since the slide preparation differs for each dye, the tissue slides are being deformed and a nonrigid registration is required before further processing. The registration of histology images is complicated because of: (i) a high resolution of histology images, (ii) complex, large, nonrigid deformations, (iii) difference in the appearance and partially missing data due to the use of multiple dyes. In this work, we propose a multistep, automatic, nonrigid image registration method dedicated to histology samples acquired with multiple stains. The proposed method consists of a feature-based affine registration, an exhaustive rotation alignment, an iterative, intensity-based affine registration, and a nonrigid alignment based on modality independent neighbourhood descriptor coupled with the Demons algorithm. A dedicated failure detection mechanism is proposed to make the method fully automatic, without the necessity of any manual interaction. The described method was proposed by the AGH team during the Automatic Non-rigid Histological Image Registration (ANHIR) challenge. The ANHIR dataset consists of 481 image pairs annotated by histology experts. Moreover, the ANHIR challenge submissions were evaluated using an independent, server-side evaluation tool. The main evaluation criteria was the target registration error normalized by the image diagonal. The median of median target registration error is below 0.19%. The proposed method is currently the second-best in terms of the average ranking of median target registration error, without statistically significant differences compared to the top-ranked method. We provide an open access to the method software and used parameters, making the results fully reproducible.
Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators

Invertible neural networks based on coupling flows (CF-INNs) have various machine learning applications such as image synthesis and representation learning. However, their desirable characteristics such as analytic invertibility come at the cost of restricting the functional forms. This poses a question on their representation power: are CF-INNs universal approximators for invertible functions? Without a universality, there could be a well-behaved invertible transformation that the CF-INN can never approximate, hence it would render the model class unreliable. We answer this question by showing a convenient criterion: a CF-INN is universal if its layers contain affine coupling and invertible linear functions as special cases. As its corollary, we can affirmatively resolve a previously unsolved problem: whether normalizing flow models based on affine coupling can be universal distributional approximators. In the course of proving the universality, we prove a general theorem to show the equivalence of the universality for certain diffeomorphism classes, a theoretical insight that is of interest by itself.
Towards Minimax Optimal Reinforcement Learning in Factored Markov Decision Processes

We study minimax optimal reinforcement learning in episodic factored Markov decision processes (FMDPs), which are MDPs with conditionally independent transition components. Assuming the factorization is known, we propose two model-based algorithms. The first one achieves minimax optimal regret guarantees for a rich class of factored structures, while the second one enjoys better computational complexity with a slightly worse regret. A key new ingredient of our algorithms is the design of a bonus term to guide exploration. We complement our algorithms by presenting several structure-dependent lower bounds on regret for FMDPs that reveal the difficulty hiding in the intricacy of the structures.
Benchmarking Web API Quality – Revisited

Modern applications increasingly interact with web APIs – reusable components, deployed and operated outside the application, and accessed over the network. Their existence, arguably, spurs application innovations, making it easy to integrate data or functionalities. While previous work has analyzed the ecosystem of web APIs and their design, little is known about web API quality at runtime. This gap is critical, as qualities including availability, latency, or provider security preferences can severely impact applications and user experience.&#x0D; In this paper, we revisit a 3-month, geo-distributed benchmark of popular web APIs, originally performed in 2015. We repeat this benchmark in 2018 and compare results from these two benchmarks regarding availability and latency. We furthermore introduce new results from assessing provider security preferences, collected both in 2015 and 2018, and results from our attempts to reach out to API providers with the results from our 2015 experiments. Our extensive experiments show that web API qualities vary 1.) based on the geo-distribution of clients, 2.) during our individual experiments, and 3.) between the two experiments. Our findings provide evidence to foster the discussion around web API quality, and can act as a basis for the creation of tools and approaches to mitigate quality issues.
CapWAP: Image Captioning with a Purpose

The traditional image captioning task uses generic reference captions to provide textual information about images. Different user populations, however, will care about different visual aspects of images. In this paper, we propose a new task, Captioning with A Purpose (CapWAP). Our goal is to develop systems that can be tailored to be useful for the information needs of an intended population, rather than merely provide generic information about an image. In this task, we use question-answer (QA) pairs—a natural expression of information need—from users, instead of reference captions, for both training and post-inference evaluation. We show that it is possible to use reinforcement learning to directly optimize for the intended information need, by rewarding outputs that allow a question answering model to provide correct answers to sampled user questions. We convert several visual question answering datasets into CapWAP datasets, and demonstrate that under a variety of scenarios our purposeful captioning system learns to anticipate and fulfill specific information needs better than its generic counterparts, as measured by QA performance on user questions from unseen images, when using the caption alone as context.
Motion estimation for ultrasound image sequences using deep learning

Increased number of medical images being used for medical praxis makes automatic processing of images a necessity. Conventional techniques of motion estimation in Ultrasound images such as exhaustive search-based block matching (ES-BM) are known to be computationally expensive and are unsuitable for portable devices. On the other hand, research in computer vision has helped the development of deep learning-based techniques for real-time motion estimation of day-to-day non-medical objects. In this paper, we propose to adopt one such deep neural network-based Fully Convolutional Siamese tracker along with Linear Kalman Filter (SiamFC-LKF) to track regions of interest in ultrasound image sequences. Siamese networks use two convolutional neural networks to create a feature map of the given reference block and the possible candidate blocks of subsequent frames. The candidate block of the subsequent frame with the maximum correlation value is considered as the tracked output. Since SiamFC does not consider any motion model, we introduce a linear Kalman filter to track the wall of the carotid artery. SiamFC-LKF and ES-BM were tested on five different image sequences of the longitudinal section of the carotid artery. Our experiments showed that SiamFC-LKF was 6 times faster and performed better than ES-BM in most of the cases.
COVID-19 and Digital Contact Tracing: Regulating the Future of Public Health Surveillance

Download This Paper Open PDF in Browser Add Paper to My Library Share: Permalink Using these links will ensure access to this page indefinitely Copy URL Copy DOI
Feature Model-Guided Online Reinforcement Learning for Self-Adaptive Services

A self-adaptive service can maintain its QoS requirements in the presence of dynamic environment changes. To develop a self-adaptive service, service engineers have to create self-adaptation logic encoding when the service should execute which adaptation actions. However, developing self-adaptation logic may be difficult due to design time uncertainty; e.g., anticipating all potential environment changes at design time is in most cases infeasible. Online reinforcement learning addresses design time uncertainty by learning suitable adaptation actions through interactions with the environment at runtime. To learn more about its environment, reinforcement learning has to select actions that were not selected before, which is known as exploration. How exploration happens has an impact on the performance of the learning process. We focus on two problems related to how a service’s adaptation actions are explored: (1) Existing solutions randomly explore adaptation actions and thus may exhibit slow learning if there are many possible adaptation actions to choose from. (2) Existing solutions are unaware of service evolution, and thus may explore new adaptation actions introduced during such evolution rather late. We propose novel exploration strategies that use feature models (from software product line engineering) to guide exploration in the presence of many adaptation actions and in the presence of service evolution. Experimental results for a self-adaptive cloud management service indicate an average speed-up of the learning process of 58.8% in the presence of many adaptation actions, and of 61.3% in the presence of service evolution. The improved learning performance in turn led to an average QoS improvement of 7.8% and 23.7% respectively .
Analysis of Position Estimation Techniques in a Surgical EM Tracking System

Electromagnetic tracking systems (EMTSs) are widely used in surgical navigation, allowing to improve the outcome of diagnosis and surgical interventions in terms of shorter hospitalization times, reduction of wound pain and increased accuracy, by providing the surgeon with real-time position of surgical instruments during medical procedures. Particular effort must be made on the development of an efficient and robust algorithm to obtain an accurate estimation of the instrument position, for distances from the magnetic field generator beyond 500 mm, which represents the maximum measurement range of current commercial systems, which strongly affects the freedom of movement of the medical team. In this paper we describe two position estimation techniques, based on a model of the magnetic field, then we analyze the robustness to errors and distortions in a set of simulations for different conditions, discussing their pros and cons in a large measurement volume, and we propose a solution by combining the two techniques. The performance of the new technique is tested on experimental data from a novel EMTS prototype, developed to increase the measurement range of current EMTSs, at a mean distance of 600 mm, and a calibration procedure is performed to reduce systematic errors. A mean position and orientation accuracy of 2.2 mm and 0.5° is achieved, along with mean repeatability errors of 1.5 mm and 0.25°, within the specifications required in many surgical applications.
Effect of ADC parameters on neural network based chaotic optical communication

The performance of traditional chaotic optical communications depends on the matching parameters of the chaotic transmitter and receiver. The chaotic receiver based on the neural network (NN) can solve the dependence on matching parameters, but the analog-to-digital converter (ADC) has an effect on the decryption performance. In this paper, the effects of sampling rate and digitalizing bit for ADC on NN-based chaotic optical communication systems with different feedback strengths at different bit rates are numerically studied. The results show that the requirements for ADC will be higher if the feedback becomes stronger. In order to achieve higher speed chaotic optical communications, the ADC with higher performance is needed. These results can give guidance for the applications of NN-based high-speed chaotic optical communication systems.
A multi-phased statistical learning based classification for network traffic

Application Traffic Identification is an imperative device for sorting out the system as it is the most popular approach to distinguish and characterize the network traffic created from different applications. The classification using conventional Port-based and Payload-based techniques has become counterproductive due to inconsistencies. However, in recent times, approaches with machine learning and statistical techniques have guaranteed higher accuracy. However, learning techniques are inadequate for solving problems with Time and Memory intricacies in vast datasets. Hence, the proposed paper presents a novel scheme of Statistical based traffic classification named Multi-Phased Statistical Based Classification methodology that renders Semi-supervised machines with advanced K-medoid clustering and C5.0 Classification algorithm. The proposed system displays a classic competence in observing the known and unknown application flows by statistical features utilization scheme that enhances the classification preciseness. Further, the trial results show that the proposed work outperforms previous approaches by achieving a higher granularity of 98–99% and reducing complexities. Ultimately, the new proposed work is evaluated on our campus traffic traces (AU-IDS). It is proven that the proposed approach accomplishes a higher exactness rate and thus encourages its implementation in real-time.
Remote sensing image retrieval by integrating automated deep feature extraction and handcrafted features using curvelet transform

Deep learning techniques have become increasingly popular for classifying large-scale image and video data. Remote sensing applications require robust search engines to retrieve similar information dependent on an example-based query instead of a tag-based query. Deep features can be extracted automatically by training raw data without having any domain-specific knowledge. However, the training time for a massive amount of multimedia datasets is high. Training complexity is reduced using pre-trained GoogleNet weights for initial feature extraction. To fine-tune the feature vector and reduce the dimensionality, a one dimension convolutional neural network (1D-CNN) is applied. There is a loss of information while resizing the input image to a pre-trained network with an acceptable input size. We proposed a new feature set by integrating handcrafted features at detailed scales and deep features to improve the system’s efficiency. The curvelet transform was used to decompose the image into coarse and detailed scales. Haralick texture features were extracted from the detail coefficients in four directions and fused with fine-tuned deep features. The proposed feature set was assessed using standard performance metrics from the literature. The proposed technique achieved improved performance with 89% accuracy for retrieval of the top 50 relevant results.