Reviewerly/exHarmony huggingface dataset:
Generalized Unsupervised Clustering of Hyperspectral Images of Geological Targets in the Near Infrared

The application of infrared hyperspectral imagery to geological problems is becoming more popular as data become more accessible and cost-effective. Clustering and classifying spectrally similar materials is often a first step in applications ranging from economic mineral exploration on Earth to planetary exploration on Mars. Semi-manual classification guided by expertly developed spectral parameters can be time consuming and biased, while supervised methods require abundant labeled data and can be difficult to generalize. Here we develop a fully unsupervised workflow for feature extraction and clustering informed by both expert spectral geologist input and quantitative metrics. Our pipeline uses a lightweight autoencoder followed by Gaussian mixture modeling to map the spectral diversity within any image. We validate the performance of our pipeline at submillimeter-scale with expert-labelled data from the Oman ophiolite drill core and evaluate performance at meters-scale with partially classified orbital data of Jezero Crater on Mars (the landing site for the Perseverance rover). We additionally examine the effects of various preprocessing techniques used in traditional analysis of hyperspectral imagery. This pipeline provides a fast and accurate clustering map of similar geological materials and consistently identifies and separates major mineral classes in both laboratory imagery and remote sensing imagery. We refer to our pipeline as "Generalized Pipeline for Spectroscopic Unsupervised clustering of Minerals (GyPSUM)."
Novel Deep-Learning-Aided Multimodal Target Tracking

Existing interacting multiple models (IMMs) are limited by the time delay in responding to system model jumps due to the nature of the soft hand-off algorithm that interacts among subfilters. To address this issue, a novel method for deep-learning-aided localization of a multimodel system is proposed in this paper. The main contribution of the proposed algorithm is that a mode estimation network based on a bidirectional long short-term memory network (BiLSTM) is newly proposed to quickly and accurately estimate the multimodal system mode, which minimizes the delay. In addition, a federated Kalman filter with a selective reinitialization algorithm from the proposed BiLSTM is proposed for better estimation of multimodal systems. Simulation and flight test results of a UAV demonstrate that the proposed algorithm yields better localization performance than the conventional IMM algorithm because the proposed mode estimation network has fast and accurate mode detection.
Toward 3D object reconstruction from stereo images

Inferring the complete 3D shape of an object from an RGB image has shown impressive results, however, existing methods rely primarily on recognizing the most similar 3D model from the training set to solve the problem. These methods suffer from poor generalization and may lead to low-quality reconstructions for unseen objects. Nowadays, stereo cameras are pervasive in emerging devices such as dual-lens smartphones and robots, which enables the use of the two-view nature of stereo images to explore the 3D structure and thus improve the reconstruction performance. In this paper, we propose a new deep learning framework for reconstructing the 3D shape of an object from a pair of stereo images, which reasons about the 3D structure of the object by taking bidirectional disparities and feature correspondences between the two views into account. Besides, we present a large-scale synthetic benchmarking dataset, namely StereoShapeNet, containing 1,052,976 pairs of stereo images rendered from ShapeNet along with the corresponding bidirectional depth and disparity maps. Experimental results on the StereoShapeNet benchmark demonstrate that the proposed framework outperforms the state-of-the-art methods. The project page is available at https://haozhexie.com/project/stereo-3d-reconstruction.
Toward Blockchain Technology in the Energy Environment

Energy management and exchange have increasingly shifted from concentrated to hierarchical modes. Numerous issues have arisen in the decentralized energy sector, including the storage of customer data and the need to ensure data integrity, fairness, and accountability in the transaction phase. The problem is that in the field of the innovative technology of blockchain and its applications, with the energy sector still in the developmental stages, there is still a need for more research to understand the full capacity of the technology in the field. The main aim of this work was to investigate the state of the current research of blockchain technologies as well as their application within the field of energy. This work also set out to identify certain research gaps and provide a set of recommendations for future directions. Among these research gaps is the application of blockchain in decentralized storage, the integration of blockchain with artificial intelligence, and security and privacy concerns, which have not received much attention despite their importance. An analysis of fifty-seven carefully reviewed studies revealed that the emerging blockchain which provides privacy-protection technologies in cryptography and other areas that can be integrated to address users’ privacy concerns is another aspect that needs further investigation. Grid operations, economies, and customers will all learn from blockchain technology as it provides disintermediation, confidentiality, and tamper-proof transfers. Moreover, it provides innovative ways for customers and small solar generators to participate more actively in the electricity sector and to benefit from their properties. Blockchains are a rapidly evolving field of research and growth. A study of this emerging technology is necessary to increase comprehension, to educate the body of expertise on blockchains, and to realize its potential. This study recommends that future work investigates the potential application of blockchain in the energy sector as well as the challenges that face its implementation from the perspective of policy makers. This future approach will enable researchers to direct their focus to the case studies approach, which will facilitate and ease the application of blockchain technology.
Hierarchical Reasoning Network for Pedestrian Attribute Recognition

Pedestrian attribute recognition, which can benefit other tasks such as person re-identification and pedestrian retrieval, is very important in video surveillance related tasks. In this paper, we observe that the existing methods tackle this problem from the perspective of multi-label classification without considering the hierarchical relationships among the attributes. In human cognition, the attributes can be categorized according to their semantic/abstraction levels. The high-level attributes can be predicted by reasoning from the low-level and medium-level attributes, while the recognition of the low-level and medium-level attributes can be guided by the high-level attributes. Based on this attribute categorization, we propose a novel Hierarchical Reasoning Network (HR-Net), which can hierarchically predict the attributes at different abstraction levels in different stages of the network. We also propose an attribute reasoning structure to exploit the relationships among the attributes at different semantic levels. Experimental results demonstrate that the proposed network gives superior performances compared to the state-of-the-art techniques.
Combining Context-Aware Embeddings and an Attentional Deep Learning Model for Arabic Affect Analysis on Twitter

Affect analysis has recently attracted a great deal of attention due to the rapid development of online social platforms (i.e., Twitter, Facebook).Affect analysis is a part of a broader area of affective computing that aims to detect and grasp human emotions or affects within a piece of writing.Context awareness is very relevant for identifying human emotions and affects behind a piece of text.Capturing the context of a piece of text is often perceived as a challenge.In addition to the own unique features of tweets (shortness, noisiness, short length, etc.), the Arabic language is characterized by its agglutination and morphological richness.In this paper, we address the problem of Arabic affect detection (multilabel emotion classification) by combining the transformer-based model for Arabic language understanding AraBERT and an attention-based LSTM-BiLSTM deep model.AraBERT generates the contextualized embedding, and the attention-based LSTM-BiLSTM determines the label-emotion of tweets by extracting both past and future contexts considering temporal information flow in both directions.Additionally, the attention mechanism is applied to the output of LSTM-BiLSTM to emphasize different words.Our proposed approach was evaluated using the reference dataset of SemEval-2018 Task 1 (Affect in Tweets).The comprehensive results show that the proposed approach outperforms eight current state-of-the-art and baseline methods, and it achieves significant accuracy (53.82%) compared to 1 st place in SemEval2018-Task1: (Affect in Tweets) competition.In addition, our proposed model outperforms the best recently reported model in the literature, with an enhancement of 2.62% in accuracy.
Back to reality: transition techniques from short HMD-based virtual experiences to the physical world

Abstract Every Virtual Reality (VR) experience has to end at some point. While there already exist concepts to design transitions for users to enter a virtual world, their return from the physical world should be considered, as well, as it is a part of the overall VR experience. We call the latter outro-transitions . In contrast to offboarding of VR experiences, that takes place after taking off VR hardware (e.g., HMDs), outro-transitions are still part of the immersive experience. Such transitions occur more frequently when VR is experienced periodically and for only short times. One example where transition techniques are necessary is in an auditorium where the audience has individual VR headsets available, for example, in a presentation using PowerPoint slides together with brief VR experiences sprinkled between the slides. The audience must put on and take off HMDs frequently every time they switch from common presentation media to VR and back. In a such a one-to-many VR scenario, it is challenging for presenters to explore the process of multiple people coming back from the virtual to the physical world at once. Direct communication may be constrained while VR users are wearing an HMD. Presenters need a tool to indicate them to stop the VR session and switch back to the slide presentation. Virtual visual cues can help presenters or other external entities (e.g., automated/scripted events) to request VR users to end a VR session. Such transitions become part of the overall experience of the audience and thus must be considered. This paper explores visual cues as outro-transitions from a virtual world back to the physical world and their utility to enable presenters to request VR users to end a VR session. We propose and investigate eight transition techniques. We focus on their usage in short consecutive VR experiences and include both established and novel techniques. The transition techniques are evaluated within a user study to draw conclusions on the effects of outro-transitions on the overall experience and presence of participants. We also take into account how long an outro-transition may take and how comfortable our participants perceived the proposed techniques. The study points out that they preferred non-interactive outro-transitions over interactive ones, except for a transition that allowed VR users to communicate with presenters. Furthermore, we explore the presenter-VR user relation within a presentation scenario that uses short VR experiences. The study indicates involving presenters that can stop a VR session was not only negligible but preferred by our participants.
A novel representation in genetic programming for ensemble classification of human motions based on inertial signals

The use of sensing technologies and novel computational methods for automated motion detection can play a major role in improving the quality of life. Recently, researchers have become interested in employing the inertial sensor technology to record human motion signals as well as the new machine learning methods for signal-based motion detection. This manuscript proposes a novel method for human motion detection based on inertial sensors. The spatial information of a motion is first used in this method for geometric feature extraction. This manuscript also aims to introduce a novel ensemble learning approach through the genetic programing paradigm. To reduce the general complexity in the process of designing the proposed classifier, an initial population of binary trees (genes) is first created and then enhanced through genetic programing to select the best classifier. A complete experiment was conducted to evaluate the proposed ensemble classifier for the classification of inertial signals of human motions. According to the experimental results based on several well-known datasets of inertial signals, the proposed approach performed appropriately in comparison with the existing methods.
Large-Scale Graph Processing on FPGAs with Caches for Thousands of Simultaneous Misses

Efficient large-scale graph processing is crucial to many disciplines. Yet, while graph algorithms naturally expose massive parallelism opportunities, their performance is limited by the memory system because of irregular memory accesses. State-of-the-art FPGA graph processors, such as ForeGraph and FabGraph, address the memory issues by using scratchpads and regularly streaming edges from DRAM, but then they end up wasting bandwidth on unneeded data. Yet, where classic caches and scratchpads fail to deliver, FPGAs make powerful unorthodox solutions possible. In this paper, we resort to extreme nonblocking caches that handle tens of thousands of outstanding read misses. They significantly increase the ability of memory systems to coalesce multiple accelerator accesses into fewer DRAM memory requests; essentially, when latency is not the primary concern, they bring the advantages expected from a very large cache at a fraction of the cost. We prove our point with an adaptable graph accelerator running on Amazon AWS f1; our implementation takes into account all practical aspects of such a design, including the challenges involved when working with modern multidie FPGAs. Running classic algorithms (PageRank, SCC, and SSSP) on large graphs, we achieve 3× geometric mean speedup compared to state-of-the-art FPGA accelerators, 1.1-5.8 × higher bandwidth efficiency and 3.0-15.3 × better power efficiency than multicore CPUs, and we support much larger graphs than the state-of-the-art on GPUs.
Object-to-Scene: Learning to Transfer Object Knowledge to Indoor Scene Recognition

Accurate perception of the surrounding scene is helpful for robots to make reasonable judgments and behaviours. Therefore, developing effective scene representation and recognition methods are of significant importance in robotics. Currently, a large body of research focuses on developing novel auxiliary features and networks to improve indoor scene recognition ability. However, few of them focus on directly constructing object features and relations for indoor scene recognition. In this paper, we analyze the weaknesses of current methods and propose an Object-to-Scene (OTS) method, which extracts object features and learns object relations to recognize indoor scenes. The proposed OTS first extracts object features based on the segmentation network and the proposed object feature aggregation module (OFAM). Afterwards, the object relations are calculated and the scene representation is constructed based on the proposed object attention module (OAM) and global relation aggregation module (GRAM). The final results in this work show that OTS successfully extracts object features and learns object relations from the segmentation network. Moreover, OTS outperforms the state-of-the-art methods by more than 2% on indoor scene recognition without using any additional streams. Code is publicly available at: https://github.com/FreeformRobotics/OTS.
An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human Pose Estimation

Most semi-supervised learning models are consistency-based, which leverage unlabeled images by maximizing the similarity between different augmentations of an image. But when we apply them to human pose estimation that has extremely imbalanced class distribution, they often collapse and predict every pixel in unlabeled images as background. We find this is because the decision boundary passes the high-density areas of the minor class so more and more pixels are gradually misclassified as background. In this work, we present a surprisingly simple approach to drive the model to learn in the correct direction. For each image, it composes a pair of easy-hard augmentations and uses the more accurate predictions on the easy image to teach the network to learn pose information of the hard one. The accuracy superiority of teaching signals allows the network to be "monotonically" improved which effectively avoids collapsing. We apply our method to the state-of-the-art pose estimators and it further improves their performance on three public datasets.
SATORI: Efficient and Fair Resource Partitioning by Sacrificing Short-Term Benefits for Long-Term Gains

Multi-core architectures have enabled data centers to increasingly co-locate multiple jobs to improve resource utilization and lower the operational cost. Unfortunately, naively co-locating multiple jobs may lead to only a modest increase in system throughput. Worse, some users may observe proportionally higher performance degradation compared to other users co-located on the same physical multi-core system. SATORI is a novel strategy to partition multi-core architectural resources to achieve two conflicting goals simultaneously: increasing system throughput and achieving fairness among the co-located jobs.
The residual generator: An improved divergence minimization framework for GAN

GAN is a generative modelling framework which has been proven as able to minimise various types of divergence measures under an optimal discriminator. However, there is a gap between the loss function of GAN used in theory and in practice. In theory, the proof of the Jensen divergence minimisation involves the min-max criterion, but in practice the non-saturating criterion is instead used to avoid gradient vanishing. We argue that the formulation of divergence minimization via GAN is biased and may yield a poor convergence of the algorithm. In this paper, we propose the Residual Generator for GAN (Rg-GAN), which is inspired by the closed-loop control theory, to bridge the gap between theory and practice. Rg-GAN minimizes the residual between the loss of the generated data to be real and the loss of the generated data to be fake from the perspective of the discriminator. In this setting, the loss terms of the generator depend only on the generated data and therefore contribute to the optimisation of the model. We formulate the residual generator for standard GAN and least-squares GAN and show that they are equivalent to the minimisation of reverse-KL divergence and a novel instance of f-divergence, respectively. Furthermore, we prove that Rg-GAN can be reduced to Integral Probability Metrics (IPMs) GANs (e.g., Wasserstein GAN) and bridge the gap between IPMs and f-divergence. Additionally, we further improve on Rg-GAN by proposing a loss function for the discriminator that has a better discrimination ability. Experiments on synthetic and natural images data sets show that Rg-GAN is robust to mode collapse, and improves the generation quality of GAN in terms of FID and IS scores.
Learning an SAR Image Despeckling Model Via Weighted Sparse Representation

Synthetic aperture radar (SAR) images are inherently degraded by the speckle noise due to the coherent imaging, which may affect the performance of subsequent image analysis task. To address this problem, a weighted sparse representation-based method is proposed in this article for SAR image despeckling. The homomorphic transformation is first adopted to convert multiplicative noise into additive one. Second, similar patches are grouped together to learn the adaptive dictionaries and sparse coefficients based on nonlocal self-similarity constraint. Moreover, weighted regularizations are adopted for coefficients to boost the performance. Finally, despeckling images are obtained via exponential transformation. Experimental results on synthetic and real-world SAR images demonstrate that our proposed method outperforms several state-of-the-art methods in terms of both quantitative measurements and visual quality.
VREUD - An End-User Development Tool to Simplify the Creation of Interactive VR Scenes

Recent advances in Virtual Reality (VR) technology and the increased availability of VR-equipped devices enable a wide range of consumer-oriented applications. For novice developers, however, creating interactive scenes for VR applications is a complex and cumbersome task that requires high technical knowledge which is often missing. This hinders the potential of enabling novices to create, modify, and execute their own interactive VR scenes. Although recent authoring tools for interactive VR scenes are promising, most of them focus on expert professionals as the target group and neglect the novices with low programming knowledge. To lower the entry barrier, we provide an open-source web-based End-User Development (EUD) tool, called VREUD, that supports the rapid construction and execution of interactive VR scenes. Concerning construction, VREUD enables the specification of the VR scene including interactions and tasks. Furthermore, VREUD supports the execution and immersive experience of the created interactive VR scenes on VR head-mounted displays. Based on a user study, we have analyzed the effectiveness, efficiency, and user satisfaction of VREUD which shows promising results to empower novices in creating their interactive VR scenes.