Reviewerly/exHarmony huggingface dataset:
Benchmarking Image Retrieval for Visual Localization

Visual localization, i.e., camera pose estimation in a known scene, is a core component of technologies such as autonomous driving and augmented reality. State-of-the-art localization approaches often rely on image retrieval techniques for one of two tasks: (1) provide an approximate pose estimate or (2) determine which parts of the scene are potentially visible in a given query image. It is common practice to use state-of-the-art image retrieval algorithms for these tasks. These algorithms are often trained for the goal of retrieving the same landmark under a large range of viewpoint changes. However, robustness to viewpoint changes is not necessarily desirable in the context of visual localization. This paper focuses on understanding the role of image retrieval for multiple visual localization tasks. We introduce a benchmark setup and compare state-of-the-art retrieval representations on multiple datasets. We show that retrieval performance on classical landmark retrieval/recognition tasks correlates only for some but not all tasks to localization performance. This indicates a need for retrieval approaches specifically designed for localization tasks. Our benchmark and evaluation protocols are available at https://github.com/naver/kapture-localization.
Ontology of Education Using Blockchain: Time Based Protocol

Education is a business process that has been a pillar of government for centuries in building the nation. The method of organizing knowledge globally does not change according to the times. Various problems currently faced by education service providers, especially at the tertiary level, in this case, call it the problem of authentication and certificate verification, which are the micro and macro outputs of a learning process. People are getting lighthouses about blockchain, the solution to educational polemic problems. The combination of science between education and blockchain technology will create a new revolution in the education industry that is felt to be stagnant and incompatible with current developments. There are various kinds of insights from this solution. Still, it is not enough to explain the ontology of education using blockchain, which, as a whole, discusses the existence of blockchain for education based on scientific studies. Ontology for knowledge will be useful as a literature platform for young researchers, especially in universities, to start basic and applied research that leads to blockchain in education. Besides, blockchain technology has weaknesses and inefficiencies, namely in terms of cost, storage space, and speed. Solutions will be proposed through this research regarding on-chain and off-chain protocols in Time Based Protocol (TBP), which is part of Blockchain technology. As a result, this research can also become the foundation for new platforms and frameworks as a solution for the government in launching blockchain-based education, which is a pioneer in creating a blockchain-based government.
Fading memory echo state networks are universal

Echo state networks (ESNs) have been recently proved to be universal approximants for input/output systems with respect to various Lp-type criteria. When 1≤p<∞, only p-integrability hypotheses need to be imposed, while in the case p=∞ a uniform boundedness hypotheses on the inputs is required. This note shows that, in the last case, a universal family of ESNs can be constructed that contains exclusively elements that have the echo state and the fading memory properties. This conclusion could not be drawn with the results and methods available so far in the literature.
Self-Attention Networks for Code Search

Developers tend to search and reuse code snippets from a large-scale codebase when they want to implement some functions that exist in the previous projects, which can enhance the efficiency of software development. As the first deep learning-based code search model, DeepCS outperforms prior models such as Sourcere and CodeHow. However, it utilizes two separate LSTM to represent code snippets and natural language descriptions respectively, which ignores semantic relations between code snippets and their descriptions. Consequently, the performance of DeepCS falls into the bottleneck, and thus our objective is to break this bottleneck. We propose a self-attention joint representation learning model, named SAN-CS (Self-Attention Network for Code Search). Comparing with DeepCS, we directly utilize the self-attention network to construct our code search model. By a weighted average operation, self-attention networks can fully capture the contextual information of code snippets and their descriptions. We first utilize two individual self-attention networks to represent code snippets and their descriptions, respectively, and then we utilize the self-attention network to conduct an extra joint representation network for code snippets and their descriptions, which can build semantic relationships between code snippets and their descriptions. Therefore, SAN-CS can break the performance bottleneck of DeepCS. We evaluate SAN-CS on the dataset shared by Gu et al. and choose two baseline models, DeepCS and CARLCS-CNN. Experimental results demonstrate that SAN-CS achieves significantly better performance than DeepCS and CARLCS-CNN. In addition, SAN-CS has better execution efficiency than DeepCS at the training and testing phase. This paper proposes a code search model, SAN-CS. It utilizes the self-attention network to perform the joint attention representations for code snippets and their descriptions, respectively. Experimental results verify the effectiveness and efficiency of SAN-CS.
Poverty Classification Using Machine Learning: The Case of Jordan

The scope of this paper is focused on the multidimensional poverty problem in Jordan. Household expenditure and income surveys provide data that are used for identifying and measuring the poverty status of Jordanian households. However, carrying out such surveys is hard, time consuming, and expensive. Machine learning could revolutionize this process. The contribution of this work is the proposal of an original machine learning approach to assess and monitor the poverty status of Jordanian households. This approach takes into account all the household expenditure and income surveys that took place since the early beginning of the new millennium. This approach is accurate, inexpensive, and makes poverty identification cheaper and much closer to real-time. Data preprocessing and handling imbalanced data are major parts of this work. Various machine learning classification models are applied. The LightGBM algorithm has achieved the best performance with 81% F1-Score. The final machine learning classification model could transform efforts to track and target poverty across the country. This work demonstrates how powerful and versatile machine learning can be, and hence, it promotes for adoption across many domains in both the private sector and government.
Distribution of geothermal arsenic in relation to geothermal play types: A global review and case study from the Anatolian plate (Turkey)

Arsenic has a natural cycle as it travels underground. It can mix with geothermal fluid in different ways under the control of magmatic and tectonic processes. Geogenic arsenic is present in many geothermal fields in the world at concentrations above the limits set for human health. The arsenic content of geothermal fluids is also related to the concept of geothermal play type, which forms geothermal systems, because the natural processes that form the geothermal system also control the arsenic cycle. In this study, an attempt is made to explain the relationship between the geothermal play type concept and geothermal arsenic circulation. For this purpose, geothermal field examples are given from around the world and Turkey. The result shows that arsenic concentrations can reach significant levels along with plate tectonic boundaries in the world. When arsenic concentrations were evaluated, the effect of major faults on the Anatolian Plate was clearly seen. Also, in the Anatolian plate where volcano-sedimentary units are common, geothermal fluids caused more effective alteration along with structural control and increased arsenic concentrations in geothermal systems. This interaction between structural elements, geothermal fluid, and the arsenic cycle shows that the concept of play type in geothermal systems should also be taken into consideration. It was determined that the places with high arsenic values are located within the convective-non-magmatic extensional geothermal play types such as Western Anatolian Extensional System and the North Anatolian Fault. The concept of play type in geothermal systems includes all systematic and external factors that make up these processes. For this reason, it is very important to evaluate the play type classification together with the arsenic cycle.
GRIF Net: Gated Region of Interest Fusion Network for Robust 3D Object Detection from Radar Point Cloud and Monocular Image

Robust and accurate scene representation is essential for advanced driver assistance systems (ADAS) such as automated driving. The radar and camera are two widely used sensors for commercial vehicles due to their low-cost, high-reliability, and low-maintenance. Despite their strengths, radar and camera have very limited performance when used individually. In this paper, we propose a low-level sensor fusion 3D object detector that combines two Region of Interest (RoI) from radar and camera feature maps by a Gated RoI Fusion (GRIF) to perform robust vehicle detection. To take advantage of sensors and utilize a sparse radar point cloud, we design a GRIF that employs the explicit gating mechanism to adaptively select the appropriate data when one of the sensors is abnormal. Our experimental evaluations on nuScenes show that our fusion method GRIF not only has significant performance improvement over single radar and image method but achieves comparable performance to the LiDAR detection method. We also observe that the proposed GRIF achieve higher recall than mean or concatenation fusion operation when points are sparse.
Microcanonical and finite-temperature <i>ab initio</i> molecular dynamics simulations on quantum computers

Ab initio molecular dynamics (AIMD) is a powerful tool to predict properties of molecular and condensed matter systems. The quality of this procedure is based on accurate electronic structure calculations. The development of quantum processors has shown great potential for the efficient evaluation of accurate ground and excited state energies of molecular systems, opening up new avenues for molecular dynamics simulations. In this work we address the use of variational quantum algorithms for the calculation of accurate atomic forces to be used in AIMD. In particular, we provide solutions for the alleviation of the statistical noise associated to the measurements of the expectation values of energies and forces, as well as schemes for the mitigation of the hardware noise sources (in particular, gate infidelities, qubit decoherence and readout errors). Despite the relative large error in the calculation of the potential energy, our results show that the proposed algorithms can provide reliable MD trajectories in the microcanonical (constant energy) ensemble. Further, exploiting the intrinsic noise arising from the quantum measurement process, we also propose a Langevin dynamics algorithm for the simulation of canonical, i.e., constant temperature, dynamics. Both algorithms (microcanonical and canonical) are applied to the simulation of simple molecular systems such as H2 and H3+. Finally, we also provide results for the dynamics of H2 obtained with IBM quantum computer ibmq_athens.
A Survey on Deep Learning Based Approaches for Scene Understanding in Autonomous Driving

As a prerequisite for autonomous driving, scene understanding has attracted extensive research. With the rise of the convolutional neural network (CNN)-based deep learning technique, research on scene understanding has achieved significant progress. This paper aims to provide a comprehensive survey of deep learning-based approaches for scene understanding in autonomous driving. We categorize these works into four work streams, including object detection, full scene semantic segmentation, instance segmentation, and lane line segmentation. We discuss and analyze these works according to their characteristics, advantages and disadvantages, and basic frameworks. We also summarize the benchmark datasets and evaluation criteria used in the research community and make a performance comparison of some of the latest works. Lastly, we summarize the review work and provide a discussion on the future challenges of the research domain.
The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima

Despite tremendous success of the stochastic gradient descent (SGD) algorithm in deep learning, little is known about how SGD finds generalizable solutions at flat minima of the loss function in high-dimensional weight space. Here, we investigate the connection between SGD learning dynamics and the loss function landscape. A principal component analysis (PCA) shows that SGD dynamics follow a low-dimensional drift-diffusion motion in the weight space. Around a solution found by SGD, the loss function landscape can be characterized by its flatness in each PCA direction. Remarkably, our study reveals a robust inverse relation between the weight variance and the landscape flatness in all PCA directions, which is the opposite to the fluctuation-response relation (aka Einstein relation) in equilibrium statistical physics. To understand the inverse variance-flatness relation, we develop a phenomenological theory of SGD based on statistical properties of the ensemble of minibatch loss functions. We find that both the anisotropic SGD noise strength (temperature) and its correlation time depend inversely on the landscape flatness in each PCA direction. Our results suggest that SGD serves as a landscape-dependent annealing algorithm. The effective temperature decreases with the landscape flatness so the system seeks out (prefers) flat minima over sharp ones. Based on these insights, an algorithm with landscape-dependent constraints is developed to mitigate catastrophic forgetting efficiently when learning multiple tasks sequentially. In general, our work provides a theoretical framework to understand learning dynamics, which may eventually lead to better algorithms for different learning tasks.
Local Correspondence Network for Weakly Supervised Temporal Sentence Grounding

Weakly supervised temporal sentence grounding has better scalability and practicability than fully supervised methods in real-world application scenarios. However, most of existing methods cannot model the fine-grained video-text local correspondences well and do not have effective supervision information for correspondence learning, thus yielding unsatisfying performance. To address the above issues, we propose an end-to-end Local Correspondence Network (LCNet) for weakly supervised temporal sentence grounding. The proposed LCNet enjoys several merits. First, we represent video and text features in a hierarchical manner to model the fine-grained video-text correspondences. Second, we design a self-supervised cycle-consistent loss as a learning guidance for video and text matching. To the best of our knowledge, this is the first work to fully explore the fine-grained correspondences between video and text for temporal sentence grounding by using self-supervised learning. Extensive experimental results on two benchmark datasets demonstrate that the proposed LCNet significantly outperforms existing weakly supervised methods.
Themed Issue Introduction: Promises and Perils of Artificial Intelligence and Advertising

(2021). Themed Issue Introduction: Promises and Perils of Artificial Intelligence and Advertising. Journal of Advertising: Vol. 50, Themed Issue: Promises and Perils of Artificial Intelligence and Advertising, pp. 1-10.
Simple Reservoir Computing Capitalizing on the Nonlinear Response of Materials: Theory and Physical Implementations

The potential of nonlinear dynamical systems serving as reservoirs has attracted much attention for the physical realization of reservoir computing (RC). Here, we propose a hardware system working as a reservoir with one simple form of nonlinearity that reflects the intrinsic characteristics of the materials. We show that insufficient dynamics in such physical systems can perform like complex dynamical systems with the assistance of external controls. Based on the idea of spatial multiplexing, this dynamical system is studied under two frameworks. The correlation between structural adjustments of the reservoir and system performance in processing various types of task is proposed. Our results are expected to enable the development of material-based devices for RC.
Multimodal Interaction Systems Based on Internet of Things and Augmented Reality: A Systematic Literature Review

Technology developments have expanded the diversity of interaction modalities that can be used by an agent (either a human or machine) to interact with a computer system. This expansion has created the need for more natural and user-friendly interfaces in order to achieve effective user experience and usability. More than one modality can be provided to an agent for interaction with a system to accomplish this goal, which is referred to as a multimodal interaction (MI) system. The Internet of Things (IoT) and augmented reality (AR) are popular technologies that allow interaction systems to combine the real-world context of the agent and immersive AR content. However, although MI systems have been extensively studied, there are only several studies that reviewed MI systems that used IoT and AR. Therefore, this paper presents an in-depth review of studies that proposed various MI systems utilizing IoT and AR. A total of 23 studies were identified and analyzed through a rigorous systematic literature review protocol. The results of our analysis of MI system architectures, the relationship between system components, input/output interaction modalities, and open research challenges are presented and discussed to summarize the findings and identify future research and development avenues for researchers and MI developers.
Behavior analysis and blockchain based trust management in VANETs

The development of vehicular ad hoc networks (VANETs) is facing great challenges. Due to the open environment in VANETs, the false information sent by malicious vehicles not only affects the fairness of information interaction but also seriously threatens the driving safety of normal vehicles. Therefore, the study of trust evaluation and management in VANETs has become hot topics in recent years. In this paper, we propose a trust management model of VANETs based on blockchain. In this model, a hidden markov model (HMM) based vehicle trust evaluation method that improve the accuracy on the detection of malicious behavior is proposed. Besides, a trust management method based on the alliance chain is designed, which greatly improves the efficiency of trust updating and querying on the premise of security. Simulation results show that the model is promising and feasible, effective in the aspects of trust evaluation and trust management.
Nonconventional Computer Arithmetic Circuits, Systems and Applications

Arithmetic plays a major role in a computer?s performance and efficiency. Building new computing platforms supported by the traditional binary arithmetic and silicon-based technologies to meet the requirements of today?s applications is becoming increasingly more challenging, regardless whether we consider embedded devices or high-performance computers. As a result, a significant amount of research effort has been devoted to the study of nonconventional number systems to investigate more efficient arithmetic circuits and improved computer technologies to facilitate the development of computational units that can meet the requirements of applications in emergent domains. This paper presents an overview of the state of the art in nonconventional computer arithmetic. Several different alternative computing models and emerging technologies are analyzed, such as nanotechnologies, superconductor devices, and biological- and quantum-based computing, and their applications to multiple domains are discussed. A comprehensive approach is followed in a survey of the logarithmic and residue number systems, the hyperdimensional and stochastic computation models, and the arithmetic for quantum- and DNA-based computing systems and techniques for approximate computing. Technologies, processors and systems addressing these nonconventional computer arithmetic systems are also reviewed, taking into consideration some of the most prominent applications, such as deep learning or postquantum cryptography. In the end, some conclusions are drawn, and directions for future research on nonconventional computer arithmetic are discussed.
Cascaded Regression Neural Nets for Kidney Localization and Segmentation-free Volume Estimation

Kidney volume is an essential biomarker for a number of kidney disease diagnoses, for example, chronic kidney disease. Existing total kidney volume estimation methods often rely on an intermediate kidney segmentation step. On the other hand, automatic kidney localization in volumetric medical images is a critical step that often precedes subsequent data processing and analysis. Most current approaches perform kidney localization via an intermediate classification or regression step. This paper proposes an integrated deep learning approach for (i) kidney localization in computed tomography scans and (ii) segmentation-free renal volume estimation. Our localization method uses a selection-convolutional neural network that approximates the kidney inferior-superior span along the axial direction. Cross-sectional (2D) slices from the estimated span are subsequently used in a combined sagittal-axial Mask-RCNN that detects the organ bounding boxes on the axial and sagittal slices, the combination of which produces a final 3D organ bounding box. Furthermore, we use a fully convolutional network to estimate the kidney volume that skips the segmentation procedure. We also present a mathematical expression to approximate the `volume error' metric from the `Sørensen-Dice coefficient.' We accessed 100 patients' CT scans from the Vancouver General Hospital records and obtained 210 patients' CT scans from the 2019 Kidney Tumor Segmentation Challenge database to validate our method. Our method produces a kidney boundary wall localization error of ~2.4mm and a mean volume estimation error of ~5%.
Multi-task faster R-CNN for nighttime pedestrian detection and distance estimation

Distance estimation and pedestrian detection are critical for safe driving operation decision-making and autonomous vehicle intelligent control strategies. This paper proposes a novel multi-task Faster R-CNN detector which simultaneously realizes distance estimation and pedestrian detection using an improved ResNet-50 architecture. Images were acquired using a near-infrared camera with two near-infrared fill-lights devices during real road nighttime scenarios. Ground truth pedestrian distances used for training were obtained using LIDAR. The data used to optimize the multi-task Faster R-CNN detector were approximately 20 k high-quality near-infrared images with marked pedestrians and tagged distance values. The proposed algorithm including the distance estimation runs at a speed exceeding 7 fps. Pedestrian detection accuracy reached nearly 80% with a total average absolute distance estimation error rate of less than 5%.