Reviewerly/exHarmony huggingface dataset:
Retrieval and Localization with Observation Constraints

Accurate visual re-localization is very critical to many artificial intelligence applications, such as augmented reality, virtual reality, robotics and autonomous driving. To accomplish this task, we propose an integrated visual re-localization method called RLOCS by combining image retrieval, semantic consistency and geometry verification to achieve accurate estimations. The localization pipeline is designed as a coarse-to-fine paradigm. In the retrieval part, we cascade the architecture of ResNet101-GeM-ArcFace and employ DBSCAN followed by spatial verification to obtain a better initial coarse pose. We design a module called observation constraints, which combines geometry information and semantic consistency for filtering outliers. Comprehensive experiments are conducted on open datasets, including retrieval on R-Oxford5k and R-Paris6k, semantic segmentation on Cityscapes, localization on Aachen Day-Night and InLoc. By creatively modifying separate modules in the total pipeline, our method achieves many performance improvements on the challenging localization benchmarks.
Fast, High-Quality Hierarchical Depth-Map Super-Resolution

The low spatial resolution of acquired depth maps is a major drawback of most RGBD sensors. However, there are many scenarios in which fast acquisition of high-resolution and high-quality depth maps would be desirable. One approach to achieve higher quality depth maps is through super-resolution. However, edge preservation is challenging, and artifacts such as depth confusion and blurring are easily introduced near boundaries. In view of this, we propose a method for fast, high-quality hierarchical depth-map super-resolution (HDS). In our method, a high-resolution RGB image is degraded layer by layer to guide the bilateral filtering of the depth map. To improve the upsampled depth map quality, we construct a feature-based bilateral filter (FBF) for the interpolation, by using the extracted RGB shallow and multi-layer features. To accelerate the process, we perform filtering only near depth boundaries and through matrix operations. We also propose an extension of our HDS model to a Classification-based Hierarchical Depth-map Super-resolution (C-HDS) model, where a context-aware trilateral filter reduces the contributions of unreliable neighbors to the current missing depth location. Experimental results show that the proposed method is significantly faster than existing methods for generating high-resolution depth maps, while also significantly improving depth quality compared to the current state-of-the-art approaches, especially for large-scale 16x super-resolution.
Furnace Temperature Prediction Using Optimized Kernel Extreme Learning Machine

In recent years, the combustion furnace has been widely applied in different fields of industrial technology, and the accurate prediction of furnace temperature can effectively help operators adjust combustion strategies to improve combustion utilization. However, furnace combustion is an extremely complex process, and its temperature change is affected by many related factors. To effectively predict the furnace temperature, a novel furnace temperature prediction method using optimized kernel extreme learning machine (OKELM) is proposed. Firstly, the optimized kernel extreme learning machine is used to establish the relationship between the furnace temperature and its related factors. Based on this, the continuous human learning optimization (CHLO) algorithm is adopted to optimize the kernel parameter and regularization coefficient, then the best OKELM with optimal parameters is adopted to predict the furnace temperature more precisely and effectively. Finally, the experiment results show that the proposed method outperforms state-of-the-art furnace temperature prediction approaches, providing high prediction accuracy and a low false prediction error.
A Vision-based Framework for Intersection Monitoring and Signal Evaluation

This work presents a typical framework for traffic measurement estimation at intersections which can assist us with traffic signals evaluation through collaboration of the computer vision techniques and traffic simulators. The proposed system is a deep-visual tracking pipeline which generates traffic measurements at different time frames. The off-the-shelf YOLO object detection architecture variants are cascaded with a discriminative correlation filter (CSRT) within the system for tracking and producing long term trajectories of road users including vehicles and pedestrians. Experimental evaluation was performed on three different intersections and YOLOv4 demonstrated better performance in pedestrian detection and traffic measurement estimation. As an application of the proposed system, traffic measurements including frame level count, turning movement counts and waiting time can be further incorporated into a traffic simulator to investigate intersection signals with current timer settings and find an appropriate signal algorithm which minimizes waiting time.
Multiresolution Representations for Large-Scale Terrain with Local Gaussian Process Regression

To address the problem of building accurate and coherent models for large-scale terrains from incomplete and noisy sensor data, this paper proposes a novel framework that can efficiently infer terrain structures by divisionally providing the best linear unbiased estimates for the elevation values. To avoid data ambiguity caused by the uncertainty of sensor data, the proposed method introduces elevation filtering to extract the terrain surfaces, which reduces the amount of data greatly while the contained terrain information is basically unchanged. Then, for the large-scale terrains, the Gaussian mixture model is used to divide the interested regions, which remarkably improves the prediction accuracy and speed. Finally, for each subregion, a gaussian process regression model based on the static kernel is used to create a multiresolution terrain representation, which can deal with incompleteness of sensor data by considering the spatial correlations of the terrain. Evaluations of the proposed technique were conducted on diverse large-scale field terrains, including the quarry, planetary emulation terrain and highland, showing that the proposed method outperforms the state-of-art terrain modeling techniques in terms of the prediction accuracy, computation speed and memory consumption. As a practical application, the path planning problem was explored based on this terrain modeling technique to produce a better path.
Sentiment analysis in aspect term extraction for mobile phone tweets using machine learning techniques

Purpose This paper aims to focus on extracting aspect terms on mobile phone (iPhone and Samsung) tweets using NLTK techniques on multiple aspect extraction is one of the challenges. Then, also machine learning techniques are used that can be trained on supervised strategies to predict and classify sentiment present in mobile phone tweets. This paper also presents the proposed architecture for the extraction of aspect terms and sentiment polarity from customer tweets. Design/methodology/approach In the aspect-based sentiment analysis aspect, term extraction is one of the key challenges where different aspects are extracted from online user-generated content. This study focuses on customer tweets/reviews on different mobile products which is an important form of opinionated content by looking at different aspects. Different deep learning techniques are used to extract all aspects from customer tweets which are extracted using Twitter API. Findings The comparison of the results with traditional machine learning methods such as random forest algorithm, K-nearest neighbour and support vector machine using two data sets iPhone tweets and Samsung tweets have been presented for better accuracy. Originality/value In this paper, the authors have focused on extracting aspect terms on mobile phone (iPhone and Samsung) tweets using NLTK techniques on multi-aspect extraction is one of the challenges. Then, also machine learning techniques are used that can be trained on supervised strategies to predict and classify sentiment present in mobile phone tweets. This paper also presents the proposed architecture for the extraction of aspect terms and sentiment polarity from customer tweets.
Cross-Modal Feature Fusion Retrieval for Remote Sensing Image-Voice Retrieval

With the increasing popularity of remote sensing technology applications, some emergency scenarios require rapid retrieval of remote sensing images, such as earthquake rescue, etc. Due to the high efficiency of voice input, researchers have focused on cross-modal remote sensing image-voice retrieval methods. However, these methods have two major drawbacks: speech input lacks discrimination and the intra-modal semantic information is under used. To address these drawbacks, we propose a novel cross-modal feature fusion retrieval model. Our model provides a more optimized cross-modal common feature space than previous models and thus optimizes the retrieval performance. First, our model adds the extra textual keyword information to the audio feature for remote sensing image retrieval. Second, it introduces inter-modality adversarial learning and intra-modality semantic discrimination into the remote sensing image-voice retrieval task. We conducted experiments on two datasets modified from the UCM-Captions dataset and the Remote Sensing Image Caption Dataset. The experimental results show that our model outperforms state-of-the-art models in this task.
Research on the intelligent settlement cloud platform of electric power materials based on the electronization of blockchain VAT special invoice

This paper follows the pilot trend of VAT (Value Added Tax) special e-invoice, and explores a new mode of power material settlement. The intelligent settlement cloud platform of VAT special e-invoice is built based on blockchain and cloud computing, which completely realizes the intelligent connection of the whole business chain of national tax system, supplier, e-commerce platform, material supply system, ERP(Enterprise Resource Planning)system and financial system, real-time sharing of the data, intelligent interaction of the information. This platform will completely cancel the traditional comparison and examination among invoice, order and receiving information, and instead realize the automatic and accurate generation, reliable transmission and digital archiving of electronic invoice, which will promote the whole process of the settlement for materials acceptance to be networked and intelligent, greatly shorten payment cycle to suppliers, continuously optimize the business environment, and boost the digital process of finance and taxation.
TMF: Temporal Motion and Fusion for action recognition

Temporal motion information plays an important role in video understanding, human action recognition and other fields. Optical flow, which contains rich temporal motion information, has been widely used in many visual tasks and has achieved superior performance. However, the extraction of optical flow is time-consuming and laborious. In this paper, we propose a Temporal Motion and Fusion (TMF) module, including a motion extraction (ME) module and a temporal crossing fusion (TCF) module. The ME module can replace the traditional optical flow, establish the matching relationship between adjacent frames on the convoluted feature maps. And then extract simple and effective short-term motion information. TCF module crosses adjacent frames and fuse the information of nonadjacent video frames to realize long-term motion information modeling. Finally, the extracted motion information is fused with the appearance information captured by 2D convolution for final recognition. The experiment proved that with only a few additional parameters and calculation costs increased, our proposed lightweight model achieves state-of-the-art results on Something-Something-V1&V2 and Diving-48, and obtains competitive results on HMDB-51 and UCF-101 among the single models.
HCTree+: A workload-guided index for approximate <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" altimg="si50.svg"><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:math>NN search

The problem of approximate k-nearest neighbor search in high-dimensional space is a fundamental problem in many applications. We have observed that most existing approaches are unable to leverage the query results to improve the search performance. To address this limitation, we present a new index, called HCTree+, that aims to improve the query performance based on incoming queries and their results. First, we adopt a simple yet effective index to support efficient search. Second, incoming queries and their results are used to optimize the index dynamically for future queries. The experimental study shows that HCTree+ outperforms the state-of-the-art algorithms in terms of accuracy while achieving the desired efficiency.
What’s This? A Voice and Touch Multimodal Approach for Ambiguity Resolution in Voice Assistants

Human speech often contains ambiguity stemming from the use of demonstrative pronouns (DPs), such as "this" and "these." While we can typically decipher which objects of interest DPs are referring to based on context, modern day voice assistants (VAs – such as Google Assistant and Siri) are yet unable to process queries containing such ambiguity. For instance, to humans, a question such as "how much is this?" can be clarified through visual reference (e.g., a buyer gestures to the seller the object they would like to purchase). To bridge this gap between human and machine cognition, we built and examined a touch + voice multimodal VA prototype that enables users to select key spatial information to embed as context and query the VA. The prototype converts results of mobile, real-time object recognition and optical character recognition models into augmented reality buttons that represent features. Users can interact with and modify the selected features through a word grid. We conducted a study to investigate: 1) how touch performs as an additional modality to resolve ambiguity in queries, 2) how users use DPs when interacting with VAs, and 3) how users perceive a VA that can understand DPs. From this procedure we found that as the query becomes more complex, users prefer the multimodal VA over the standard VA without experiencing elevated cognitive load. Additionally, even though it took some time getting used to, many participants eventually became comfortable with using DPs to interact with the multimodal VA and appreciated the improved human-likeness of human-VA conversations.
Fast Replanning Multi-Heuristic A

In this paper, we proposed a novel path replanning algorithm on arbitrary graphs. To avoid computationally heavy preprocessing and to reduce required memory to store the expanded vertices of the previous search, we defined the feature vertices, which are extracted from the previous path by a simple algorithm to compare the costs between adjacent vertices along the path once. Proper additional heuristic functions are designed for these feature vertices to work as local attractors guiding the search toward the previous path’s neighbors. To avoid unnecessary expansions and speed up the search, these additional heuristic functions are properly managed to stop intriguing or guiding search toward the feature vertices. The proposed algorithm of Fast Replanning Multi-Heuristic A* is a variation of Shared Multi-Heuristic A* while removing or deactivating the additional heuristic functions during the search. Fast Replanning Multi-Heuristic A* guarantees the bounded suboptimality while efficiently exploring the graph toward the goal vertex. The performance of the proposed algorithm was compared with weighted A* and D* lite by simulating numerous path replanning problems in maze-like maps.
Learning to Recognize Human Actions from Noisy Skeleton Data via Noise Adaptation

Recent studies have made great progress on skeleton-based action recognition. However, most of them are developed with relatively clean skeletons without the presence of intensive noise. We argue that the models learned from relatively clean data are not well generalizable to handle noisy skeletons commonly appeared in the real world. In this paper, we address the challenge of recognizing human actions from noisy skeletons, which is seldom explored by previous methods. Beyond exploring the new problem, we further take a new perspective to address it, \textit{i.e.}, noise adaptation, which gets rid of explicit skeleton noise modeling and reliance on skeleton ground truths. Specifically, we develop regression-based and generation-based adaptation models according to whether pairs of noisy skeletons are available. The regression-based model aims to learn noise-suppressed intrinsic feature representations by mapping pairs of noisy skeletons into a noise-robust space. When only unpaired skeletons are accessible, the generation-based model aims to adapt the features from noisy skeletons to a low-noise space by adversarial learning. To verify our proposed model and facilitate research on noisy skeletons, we collect a new dataset Noisy Skeleton Dataset (NSD), the skeletons of which are with much noise and more similar to daily-life data than previous datasets. Extensive experiments are conducted on the NSD, VV-RGBD and N-UCLA datasets, and results consistently show the outstanding performance of our proposed model.
Resource Sharing of Smart City Based on Blockchain

The concept of smart city refers to the improvement of the quality of life of the city by making full use of idle resources by sharing. However, limited by the technical level, the current resource sharing system mostly adopts centralized data storage mode. Systems managed in this way are vulnerable to multiple threats. The tested blockchain technology with the characteristics of decentralization and tamper resistance can effectively prevent various risks. Starting with the architecture of blockchain intelligent contract, this paper puts forward a structural optimization factor model of intelligent contract. To optimize the structure of blockchain intelligent contract, the gas optimization theory is put forward by changing the order, reducing the use of costly EVM data fields, reducing redundant fields, and optimizing intelligent contract codes. Experimental analysis of the proposed model is carried out, and the effectiveness of the proposed method is verified by comparing the transaction execution time of cost calculation with the cost of executing gas, which can provide reference for the selection of intelligent contract organization structure of smart city resource sharing system.
Evaluation of Deep Learning Techniques for Content Extraction in Spanish Colonial Notary Records

Processing and analyzing historical manuscripts is considered one of the most challenging problems in the document analysis and recognition domain. Manuscripts written in cursive are even more difficult due to overlapping words with random spacing, irregular and varying characters' shapes, poor scan quality, and insufficient labeled data. Despite the significant achievements of deep learning approaches in computer vision, handwritten word recognition is far from solved. Most of the existing methods focus on well-segmented word datasets. In this paper, we present an empirical study investigating how well state-of-the-art deep learning models perform on detection and recognition of handwritten words in Spanish American notary records. Professional historians were involved in preparing a labeled dataset of 26,482 Spanish words employed in the experiments. We investigate the performance of some state-of-the-art models on optical character recognition (OCR) on handwritten text documents: Keras-OCR, the object detection algorithm "You Only Look Once" (YOLO), Tesseract OCR, Kraken, and Calamari-OCR. Since YOLO does not include a text recognizer, we propose YOLO-OCR, an innovative model to detect and recognize words in historical manuscripts written in Spanish. Our results show the performance of pre-trained models on our dataset and that Keras-OCR and YOLO-OCR models are highly valuable for content extraction.
Whole-Body Real-Time Motion Planning for Multicopters

Multicopters are able to perform high maneuverability yet their potential have not been fully achieved. In this work, we propose a full-body, optimization-based motion planning framework that takes shape and attitude of aerial robot into consideration such that the aggressiveness of drone maneuvering improves significantly in cluttered environment. Our method takes in a series of intersecting polyhedrons that describe a range of 3D free spaces and outputs a time-indexed trajectory in real-time with full-body collision-free guarantee. The drone is modeled as a tilted cuboid, yet we argue that our framework can be freely adjusted to fit multicopters of different shapes. Guaranteeing dynamic feasibility and safety conditions, our framework transforms the original constrained nonlinear programming problem to an unconstrained one in higher dimensions which is further solved by quasi-Newton methods. Benchmark has shown that our method improves the state-of-art with orders of magnitude in terms of computation time and memory usage. Simulations and onboard experiments are carried out as validation.
The Aircraft Context Dataset: Understanding and Optimizing Data Variability in Aerial Domains

Despite their increasing demand for assistant and autonomous systems, the recent shift towards data-driven approaches has hardly reached aerial domains, partly due to a lack of specific training and test data. We introduce the Aircraft Context Dataset, a composition of two inter-compatible large-scale and versatile image datasets focusing on manned aircraft and UAVs, respectively. In addition to fine-grained annotations for multiple learning tasks, we define and apply a set of relevant meta-parameters and showcase their potential to quantify dataset variability as well as the impact of environmental conditions on model performance. Baseline experiments are conducted for detection, classification and semantic labeling on multiple dataset variants. Their evaluation clearly shows that our contribution is an essential step towards overcoming the data gap and that the proposed variability concept significantly increases the efficiency of specializing models as well as continuously and purposefully extending the dataset.
A Few Shot Adaptation of Visual Navigation Skills to New Observations using Meta-Learning

Target-driven visual navigation is a challenging problem that requires a robot to find the goal using only visual inputs. Many researchers have demonstrated promising results using deep reinforcement learning (deep RL) on various robotic platforms, but typical end-to-end learning is known for its poor extrapolation capability to new scenarios. Therefore, learning a navigation policy for a new robot with a new sensor configuration or a new target still remains a challenging problem. In this paper, we introduce a learning algorithm that enables rapid adaptation to new sensor configurations or target objects with a few shots. We design a policy architecture with latent features between perception and inference networks and quickly adapt the perception network via meta-learning while freezing the inference network. Our experiments show that our algorithm adapts the learned navigation policy with only three shots for unseen situations with different sensor configurations or different target colors. We also analyze the proposed algorithm by investigating various hyperparameters.