Reviewerly/exHarmony huggingface dataset:
SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-View Stereopsis

Multi-view stereopsis (MVS) tries to recover the 3D model from 2D images. As the observations become sparser, the significant 3D information loss makes the MVS problem more challenging. Instead of only focusing on densely sampled conditions, we investigate sparse-MVS with large baseline angles since the sparser sensation is more practical and more cost-efficient. By investigating various observation sparsities, we show that the classical depth-fusion pipeline becomes powerless for the case with a larger baseline angle that worsens the photo-consistency check. As another line of the solution, we present SurfaceNet+, a volumetric method to handle the 'incompleteness' and the 'inaccuracy' problems induced by a very sparse MVS setup. Specifically, the former problem is handled by a novel volume-wise view selection approach. It owns superiority in selecting valid views while discarding invalid occluded views by considering the geometric prior. Furthermore, the latter problem is handled via a multi-scale strategy that consequently refines the recovered geometry around the region with the repeating pattern. The experiments demonstrate the tremendous performance gap between SurfaceNet+ and state-of-the-art methods in terms of precision and recall. Under the extreme sparse-MVS settings in two datasets, where existing methods can only return very few points, SurfaceNet+ still works as well as in the dense MVS setting. The benchmark and the implementation are publicly available at https://github.com/mjiUST/SurfaceNet-plus.
Hierarchical Convolutional Neural Network for Handwritten Digits Recognition

The application of a combination of convolutional neural networks for the recognition of handwritten digits is considered. Recognition is carried out by two sets of the networks following each other. The first neural network selects two digits with maximum activation functions. Depending on the winners, the next network is activated, which selects one digit from two. The proposed algorithm is tested on the data from MNIST. The minimal handwriting recognition error was estimated with this approach.
Crowd counting via scale-communicative aggregation networks

Crowd counting is a fundamental computer vision task that draws increasing attention in recent years, due to its wide applications in commercial activities and public securities. Despite much process has been achieved by applying the resurgent neural networks in this task, critical challenges still lie in tremendous variation of crowd scales, together with other issues like background clutters and occlusions, making the crowd appearances hard to model. To address these challenges, we propose a scale-communicative aggregation network (SCANet) for crowd counting. Our model is characterized by three aspects: (i) It contains different streams of convolutional neural networks (CNNs), where each stream consumes an individual scaled version of the input image and communicates complementarily to produce a high-resolution density map. (ii) Each CNN stream obtains robust feature presentation via our proposed multi-scale feature encoders (MSFEs) with dilated convolutional layers, and skip connections are adopted to exploit multi-stage feature aggregation. (iii) A multi-scale structural similarity metric along with Euclidean distance is introduced for optimizing the quality of generated density maps. Extensive experiments and comparisons on several crowd counting benchmarks demonstrate the effectiveness of our proposed method.
Audio-Processing-Based Human Detection at Disaster Sites With Unmanned Aerial Vehicle

This paper describes a human search system that uses an unmanned aerial vehicle (UAV). The use of robots to search for people is expected to become an auxiliary tool for saving lives during a disaster. In particular, because UAVs can collect information from the air, there has been much research into human search using UAVs equipped with cameras. However, the disadvantage of cameras is that they struggle to detect people who are hidden in shadows. To solve this problem, we mounted an array microphone on a UAV and to detect the human voice as a means of finding people that cameras cannot. Also a search method is proposed that combines voice and camera human detection to compensate for their respective shortcomings. The rate and accuracy of human detection by the proposed method are assessed experimentally.
Multi-Attention Multimodal Sentiment Analysis

Sentiment analysis plays an important role in natural-language processing. It has been performed on multimodal data including text, audio, and video. Previously conducted research does not make full utilization of such heterogeneous data. In this study, we propose a model of Multi-Attention Recurrent Neural Network (MA-RNN) for performing sentiment analysis on multimodal data. The proposed network consists of two attention layers and a Bidirectional Gated Recurrent Neural Network (BiGRU). The first attention layer is used for data fusion and dimensionality reduction, and the second attention layer is used for the augmentation of BiGRU to capture key parts of the contextual information among utterances. Experiments on multimodal sentiment analysis indicate that our proposed model achieves the state-of-the-art performance of 84.31% accuracy on the Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis (CMU-MOSI) dataset. Furthermore, an ablation study is conducted to evaluate the contributions of different components of the network. We believe that our findings of this study may also offer helpful insights into the design of models using multimodal data.
The relationship between features of source text use and integrated writing quality

Academic writing ability is an important aspect of success in higher education. Recently, standardized academic language proficiency tests (such as the TOEFL) have begun to include integrated writing tasks, which ask test-takers to read and/or listen to a passage and construct a response that reflects the information in the passage(s). Arguably, integrated tasks more closely resemble authentic academic tasks than independent tasks, and therefore increase the construct validity of assessment tools that include them (Cumming et al., 2005; Taylor & Angelis, 2008). A number of recent studies have investigated differences between the products and processes of responding to independent and integrated tasks (Guo et al., 2013; Kyle & Crossley, 2016; Plakans, 2009a; Plakans & Gebril, 2013). In this study, the relationship between automated indices of source text use and holistic quality scores is investigated, building on Plakans and Gebril (2013). The results indicate that a number of indices related to content word overlap and n-gram overlap explained a substantial portion of the variance in holistic scores. These results generally align with the findings of Plakans and Gebril (2013), and provide important implications for increasing the construct coverage of automated scoring models (such as e-rater).
Intelligent trademark similarity analysis of image, spelling, and phonetic features using machine learning methodologies

The rapid development of consumer products with short life spans, along with fast, global e-commerce and e-marketing distribution of products and services requires greater due diligence to protect intangible assets such as brands and corporate logos which can easily be copied or distributed through grey channels and internet sales sites. Trademarks (TMs) are government registered intellectual property rights (IPRs) used to legally protect a companies' identities and brand equity. The rapid growth of global trademark (TM) registrations and the number of TM infringement cases pose a great challenge for TM owners to detect infringement and take action to protect TMs, consumer trust, and market share. This research develops advanced TM similarity assessment models using machine learning (ML) approaches. Litigation principles over similarity follow US TM laws which are consistent with global TM protection convention under the World Intellectual Property Organization (WIPO). This research covers the similarity analysis of TM spelling, pronunciation, and images, which are most likely to cause TM confusion among customers. The research focuses on deploying machine learning for natural language (spelling and phonetic features) and image similarity analyses. The vector space modeling algorithms are trained and verified for the similarity analysis of TM wordings in both spelling and pronunciation. The convolutional neural network and Siamese neural network models are trained and verified for TM image similarity comparison. The training and testing sets consist of 250,000 and 20,000 different image pairs respectively. This research provides a significant contribution toward implementing intelligent and automated IPR protection. The system solution supports users (companies, TM attorneys, or IP officers) to identify similar registered TMs before registering new TMs ensuring uniqueness to avoid infringement disputes. The solution also supports automatic screening of online content to detect potential infringement of TM images and wording for effective global IPR protection.
Dialogue Enhanced Extended Reality: Interactive System for the Operator 4.0

The nature of industrial manufacturing processes and the continuous need to adapt production systems to new demands require tools to support workers during transitions to new processes. At the early stage of transitions, human error rate is often high and the impact in quality and production loss can be significant. Over the past years, eXtended Reality (XR) technologies (such as virtual, augmented, immersive, and mixed reality) have become a popular approach to enhance operators’ capabilities in the Industry 4.0 paradigm. The purpose of this research is to explore the usability of dialogue-based XR enhancement to ease the cognitive burden associated with manufacturing tasks, through the augmentation of linked multi-modal information available to support operators. The proposed Interactive XR architecture, using the Spoken Dialogue Systems’ modular and user-centred architecture as a basis, was tested in two use case scenarios: the maintenance of a robotic gripper and as a shop-floor assistant for electric panel assembly. In both cases, we have confirmed a high user acceptance rate with an efficient knowledge communication and distribution even for operators without prior experience or with cognitive impairments, therefore demonstrating the suitability of the solution for assisting human workers in industrial manufacturing processes. The results endorse an initial validation of the Interactive XR architecture to achieve a multi-device and user-friendly experience to solve industrial processes, which is flexible enough to encompass multiple tasks.
Explaining Deep Learning Models Through Rule-Based Approximation and Visualization

This article describes a novel approach to the problem of developing explainable machine learning models. We consider a deep reinforcement learning (DRL) model representing a highway path planning policy for autonomous highway driving [1]. The model constitutes a mapping from the continuous multidimensional state space characterizing vehicle positions and velocities to a discrete set of actions in longitudinal and lateral direction. It is obtained by applying a customized version of the double deep Q-network learning algorithm [2]. The main idea is to approximate the DRL model with a set of IF-THEN rules that provide an alternative interpretable model, which is further enhanced by visualizing the rules. This concept is rationalized by the universal approximation properties of the rule-based models with fuzzy predicates. The proposed approach includes a learning engine composed of zero-order fuzzy rules, which generalize locally around the prototypes by using multivariate function models. The adjacent (in the data space) prototypes, which correspond to the same action, are further grouped and merged into the so-called MegaClouds reducing significantly the number of fuzzy rules. The input selection method is based on ranking the density of the individual inputs. Experimental results show that the specific DRL agent can be interpreted by approximating with families of rules of different granularity. The method is computationally efficient and can be potentially extended to addressing the explainability of the broader set of fully connected deep neural network models.
Automatic Reminiscence Therapy for Dementia

With people living longer than ever, the number of cases with dementia such as Alzheimer's disease increases steadily. It affects more than 46 million people worldwide, and it is estimated that in 2050 more than 100 million will be affected. While there are no effective treatments for these terminal diseases, therapies such as reminiscence, that stimulate memories from the past are recommended. Currently, reminiscence therapy takes place in care homes and is guided by a therapist or a carer. In this work, we present an AI-based solution to automate the reminiscence therapy. This consists of a dialogue system that uses photos of the users as input to generate questions about their life. Overall, this paper presents how reminiscence therapy can be automated by using deep learning, and deployed to smartphones and laptops, making the therapy more accessible to every person affected by dementia.
DeepConversion: Voice conversion with limited parallel training data

A deep neural network approach to voice conversion usually depends on a large amount of parallel training data from source and target speakers. In this paper, we propose a novel conversion pipeline, DeepConversion, that leverages a large amount of non-parallel, multi-speaker data, but requires only a small amount of parallel training data. It is believed that we can represent the shared characteristics of speakers by training a speaker independent general model on a large amount of publicly available, non-parallel, multi-speaker speech data. Such general model can then be used to learn the mapping between source and target speaker more effectively from a limited amount of parallel training data. We also propose a strategy to make full use of the parallel data in all models along the pipeline. In particular, the parallel data is used to adapt the general model towards the source-target speaker pair to achieve a coarse grained conversion, and to develop a compact Error Reduction Network (ERN) for a fine-grained conversion. The parallel data is also used to adapt the WaveNet vocoder towards the source-target pair. The experiments show that DeepConversion that only uses a limited amount of parallel training data, consistently outperforms the traditional approaches that use a large amount of parallel training data, in both objective and subjective evaluations.
Improving the slice interaction of 2.5D CNN for automatic pancreas segmentation

Purpose Volumetric pancreas segmentation can be used in the diagnosis of pancreatic diseases, the research about diabetes and surgical planning. Since manual delineation is time‐consuming and laborious, we develop a deep learning‐based framework for automatic pancreas segmentation in three dimensional (3D) medical images. Methods A two‐stage framework is designed for automatic pancreas delineation. In the localization stage, a Square Root Dice loss is developed to handle the trade‐off between sensitivity and specificity. In refinement stage, a novel 2.5D slice interaction network with slice correlation module is proposed to capture the non‐local cross‐slice information at multiple feature levels. Also a self‐supervised learning‐based pre‐training method, slice shuffle, is designed to encourage the inter‐slice communication. To further improve the accuracy and robustness, ensemble learning and a recurrent refinement process are adopted in the segmentation flow. Results The segmentation technique is validated in a public dataset (NIH Pancreas‐CT) with 82 abdominal contrast‐enhanced 3D CT scans. Fourfold cross‐validation is performed to assess the capability and robustness of our method. The dice similarity coefficient, sensitivity, and specificity of our results are 86.21 ± 4.37%, 87.49 ± 6.38% and 85.11 ± 6.49% respectively, which is the state‐of‐the‐art performance in this dataset. Conclusions We proposed an automatic pancreas segmentation framework and validate in an open dataset. It is found that 2.5D network benefits from multi‐level slice interaction and suitable self‐supervised learning method for pre‐training can boost the performance of neural network. This technique could provide new image findings for the routine diagnosis of pancreatic disease.
Dynamic Harmonic Analysis With FIR Filters Designed With O-Splines

Splines are at the essence of signal processing. Not only in sampling and interpolation, but also in filter design, image processing, and multi-resolution analysis. A new class of splines is presented here. They are referred to as O-splines since their knots are separated by one fundamental cycle. They are used as optimal state samplers, in the sense that their coefficients provide the derivatives for the best Taylor approximation to a given signal about a time instance or the best Hermite interpolation between two of them. They are the impulse response of the filters of the Discrete-Time Taylor-Fourier Transform (DTTFT) filter bank. Lowpass O-spline coincides with the Lagrange central interpolation kernel, which converges towards the ideal Sinc function. It comes with its derivatives which in turn converge to the ideal lowpass differentiator. The bandpass O-splines are harmonic splines since they are modulations of the former kernel at harmonic frequencies. In closed-form, they reduce the computational complexity of the DTTFT and can be used to design ideal bandpass filters at a particular frequency. By increasing the order they define a ladder of spaces very useful for multi-resolution and time-frequency analysis. Examples are provided at the end of the paper. Naturally, a new family of wavelets is coming soon from these splines.
Multi-attention guided feature fusion network for salient object detection

Though with the rapid development of deep learning, salient object detection methods have achieved increasingly better performance, how to get effective feature representation to predict more accurate saliency maps is still a burning problem we need to consider. To overcome this situation, most previous works tend to focus on skip-based architecture to integrate hierarchical information of different scales and layers. However, a simple concatenation of high-level features and low-level features is not all-powerful because cluttered and noisy information can cause negative consequences. Concerning the issue mentioned above, we propose a Multi-Attention guided Feature-fusion network (MAF) which can alleviate the problem from two aspects. For one thing, we use a novel Channel-wise Attention Block (CAB) to in charge of message passing layer by layer from a global view, which utilizes the semantic cues in the higher convolutional block to instruct the feature selection in the lower block. For another, a Position Attention Block (PAB) also works on integrated features to model pixel relationships and capture rich contextual dependencies. Under the guidance of multi-attention, discriminative features are selected to conduct a new end-to-end densely supervised encoder-decoder network which detects salient objects more uniformly and precisely. As the experimental results on five benchmark datasets show, our methods perform favorably against other state-of-the-art approaches.
Scalable, Multi-Constraint, Complex-Objective Graph Partitioning

We introduce XtraPuLP, a distributed-memory graph partitioner designed to process irregular trillion-edge graphs. XtraPuLP is based on the scalable label propagation community detection technique, which has been demonstrated in various prior works as a viable means to produce high quality partitions of skewed and small-world graphs with minimal computation time. Our XtraPuLP implementation can also be generalized to compute partitions with an arbitrary number of constraints, and it can compute partitions with balanced communication load across all parts. On a collection of large sparse graphs, we show that XtraPuLP partitioning is considerably faster than state-of-the-art partitioning methods, while also demonstrating that XtraPuLP can produce partitions of real-world graphs with billion+ vertices and over a hundred billion edges in minutes. Additionally, we demonstrate XtraPuLP on a variety of applications, including large-scale graph analytics and sparse matrix-vector multiplication.
Fine-Grained Image-to-Image Transformation Towards Visual Recognition

Existing image-to-image transformation approaches primarily focus on synthesizing visually pleasing data. Generating images with correct identity labels is challenging yet much less explored. It is even more challenging to deal with image transformation tasks with large deformation in poses, viewpoints, or scales while preserving the identity, such as face rotation and object viewpoint morphing. In this paper, we aim at transforming an image with a fine-grained category to synthesize new images that preserve the identity of the input image, which can thereby benefit the subsequent fine-grained image recognition and few-shot learning tasks. The generated images, transformed with large geometric deformation, do not necessarily need to be of high visual quality but are required to maintain as much identity information as possible. To this end, we adopt a model based on generative adversarial networks to disentangle the identity related and unrelated factors of an image. In order to preserve the fine-grained contextual details of the input image during the deformable transformation, a constrained nonalignment connection method is proposed to construct learnable highways between intermediate convolution blocks in the generator. Moreover, an adaptive identity modulation mechanism is proposed to transfer the identity information into the output image effectively. Extensive experiments on the CompCars and Multi-PIE datasets demonstrate that our model preserves the identity of the generated images much better than the state-of-the-art image-to-image transformation models, and as a result significantly boosts the visual recognition performance in fine-grained few-shot learning.
A Quantum Computational Approach to Correspondence Problems on Point Sets

Modern adiabatic quantum computers (AQC) are already used to solve difficult combinatorial optimisation problems in various domains of science. Currently, only a few applications of AQC in computer vision have been demonstrated. We review AQC and derive a new algorithm for correspondence problems on point sets suitable for execution on AQC. Our algorithm has a subquadratic computational complexity of the state preparation. Examples of successful transformation estimation and point set alignment by simulated sampling are shown in the systematic experimental evaluation. Finally, we analyse the differences in the solutions and the corresponding energy values.
Distilling Knowledge for Fast Retrieval-based Chat-bots

Response retrieval is a subset of neural ranking in which a model selects a suitable response from a set of candidates given a conversation history. Retrieval-based chat-bots are typically employed in information seeking conversational systems such as customer support agents. To make pairwise comparisons between a conversation history and a candidate response, two approaches are common: cross-encoders performing full self-attention over the pair and bi-encoders encoding the pair separately. The former gives better prediction quality but is too slow for practical use. In this paper, we propose a new cross-encoder architecture and transfer knowledge from this model to a bi-encoder model using distillation. This effectively boosts bi-encoder performance at no cost during inference time. We perform a detailed analysis of this approach on three response retrieval datasets.