Reviewerly/exHarmony huggingface dataset:
Mutual Space Generation with Relative Translation Gains in Redirected Walking for Asymmetric Remote Collaboration

This study proposed a Mixed Reality (MR) mutual space gener-ation method that aligns the Virtual Reality (VR) clients' spaces around the Augmented Reality (AR) host space's reference object and adjusts the virtual space's scale with translation-based Redi-rected Walking (RDW). Although there were methods to create shared spaces by finding optimal transform or each space to max-imize the semantically aligned area, there was a lack of research on utilizing the effect of rescaling a VR client's space with the RDW technique. We leveraged Relative Translation Gains (RTGs), a length-based RDW technique, to maximize the semantic information matching ratio between the AR host space and the VR clients' spaces by rescaling the client's virtual space within a suitable threshold range according to the spatial configuration. Our pilot study results show that the semantic information matching rate between the AR and the VR space increased compared to the existing one-to-one scale matching method while maintaining a unified coordinate sys-tem. Our findings can be used to create a more expansive mutual space for asymmetric remote collaboration even if remote spaces increase and challenging to secure a sufficient shared area.
Multi-GPU Registration of High-Resolution Multispectral Images Using HSI-KAZE in a Cluster System

Feature-based registration methods have been demonstrated to be very effective to register multispectral images with large distortions or transformations despite the higher execution time that they require. In this paper, a first approach to a multi-node, multi-GPU implementation of the Hyper-spectral KAZE (HSI-KAZE) method for co-registering bands and multispectral images is presented. Different multispectral datasets are distributed among the available nodes of a cluster using MPI and exploiting the parallel stream-based capabilities of the GPUs inside each node using CUDA.
Low-Code Internet of Things Application Development for Edge Analytics

Abstract Internet of Things (IoT) applications combined with edge analytics are increasingly developed and deployed across a wide range of industries by engineers who are non-expert software developers. In order to enable them to build such IoT applications, we apply low-code technologies in this case study based on Model Driven Development. We use two different frameworks: DIME for the application design and implementation of IoT and edge aspects as well as analytics in R, and Pyrus for data analytics in Python, demonstrating how such engineers can build innovative IoT applications without having the full coding expertise. With this approach, we develop an application that connects a range of heterogeneous technologies: sensors through the EdgeX middleware platform with data analytics and web based configuration applications. The connection to data analytics pipelines can provide various kinds of information to the application users. Our innovative development approach has the potential to simplify the development and deployment of such applications in industry.
DATA: Domain-Aware and Task-Aware Self-supervised Learning

The paradigm of training models on massive data without label through self-supervised learning (SSL) and fine-tuning on many downstream tasks has become a trend recently. However, due to the high training costs and the un-consciousness of downstream usages, most self-supervised learning methods lack the capability to correspond to the diversities of downstream scenarios, as there are various data domains, different vision tasks and latency constraints on models. Neural architecture search (NAS) is one universally acknowledged fashion to conquer the issues above, but applying NAS on SSL seems impossible as there is no label or metric provided for judging model selection. In this paper, we present DATA, a simple yet effective NAS approach specialized for SSL that provides Domain-Aware and Task-Aware pre-training. Specifically, we (i) train a supernet which could be deemed as a set of millions of networks covering a wide range of model scales without any label, (ii) propose a flexible searching mechanism compatible with SSL that enables finding networks of different computation costs, for various downstream vision tasks and data domains without explicit metric provided. Instantiated With MoCo v2, our method achieves promising results across a wide range of computation costs on down-stream tasks, including image classification, object detection and semantic segmentation. DATA is orthogonal to most existing SSL methods and endows them the ability of customization on downstream needs. Extensive experiments on other SSL methods demonstrate the generalizability of the proposed method. Code is released at https://github.com/GAIA-vision/GAIA-ssl.
XAI &amp; I: Self-explanatory AI facilitating mutual understanding between AI and human experts

Traditionally, explainable artificial intelligence seeks to provide explanation and interpretability of high-performing black-box models such as deep neural networks. Interpretation of such models remains difficult, because of their high complexity. An alternative method is to instead force a deep-neural network to use human-intelligible features as the basis for its decisions. We tested this approach using the natural category domain of rock types. We compared the performance of a black-box implementation of transfer-learning using Resnet50 to that of a network first trained to predict expert-identified features and then forced to use these features to categorise rock images. The performance of this feature-constrained network was virtually identical to that of the unconstrained network. Further, a partially constrained network forced to condense down to a small number of features that was not trained with expert features did not result in these abstracted features being intelligible; nevertheless, an affine transformation of these features could be found that aligned well with expert-intelligible features. These findings show that making an AI intrinsically intelligible need not be at the cost of performance.
20 Million-Dollar Problems for Any Brain Models and a Holistic Solution: Conscious Learning

This is a theoretical paper. It raises 20 open problems each of which is estimated to require one million dollars of investment or more. They are (1) the image annotation problem (e.g., retina is without bounding box to learn, unlike ImageNet), (2) the sensorimotor recurrence problem (e.g., all big data sets are invalid), (3) the motor-supervision problem (e.g., impractical to supervise motors throughout lifetime), (4) the sensor calibration problem (e.g., a life calibrates the eyes automatically), (5) the inverse kinematics problem (e.g., a life calibrates all redundant limbs automatically), (6) the government-free problem (i.e., no task-aware homunculus inside a brain), (7) the closed-skull problem (e.g., supervising hidden neurons is biologically implausible), (8) the nonlinear controller problem (e.g., a brain is a nonlinear controller but task-nonspecific), (9) the curse of dimensionality problem (e.g., a set of global features is insufficient for a life), (10) the under-sample problem (i.e., few available examples in a life), (11) the distributed vs. local representations problem (i.e., how both representations emerge), (12) the symbol problem (also called grounding problem, thus must be free from any symbols), (13) the local minima problem (so, avoid error-backprop learning and Post-Selections), (14) the abstraction problem (i.e., require various invariances and transfers), (15) the compositionality problem (e.g., metonymy beyond those composable from sentences), (16) the smooth representations problem (e.g., brain representations are globally smooth), (17) the motivation problem (e.g., including reinforcements and various emotions), (18) the global optimality problem (e.g., avoid catastrophic memory loss and Post-Selections), (19) the auto-programming for general purposes (APFGP) problem, (20) the brain-thinking problem. The paper discusses also why the proposed holistic solution of conscious learning [1], [2] solves each.
SEIPS-based patient journey mapping: Application to a patient visit in the emergency department

Patient Journey Mapping (PJM) is an emerging method to describe complex health care processes from the patient perspective. In this paper we describe a novel PJM method that integrates the SEIPS model, in particular the five work system elements (i.e., person, tasks, tools/technology, environment, and organization). The SEIPS-based PJM method supports the integration of multiple perspectives (e.g., patient, clinician) throughout the design and development process, and helps to represent non-linear health care processes, such as those experienced by patients who present to the emergency department.
Prototypical Network for Few-Shot Signal Recognition

With the rapid development of 5G and IoT technologies, Wireless Physical Layer Identification (WPLI) is essential for securing wireless communication technology, facing with more complex communication environment and access to heterogeneous networks. How to carry out device authentication in a small sample environment is an urgent and challenging problem. However, most data-driven deep learning (DL)-based WPLI approaches currently are not suitable for scenarios with insufficient samples. To address the issue, we learn from the prototype network in the image domain and use the meta-training method to solve the Few-Shot Learning (FSL) problem in signal recognition. To compare the prototype network performance, we conduct a 1-shot/5-shot test and compare with two baseline models. We further explore the impact of cross-domain on few-shot signal recognition. Simulation results show that prototype network achieves excellent performance on the same domain few-shot classification problem, but when there is distribution drift, the prototype network model adaptability is lower than the baseline model.
AxFormer: Accuracy-driven Approximation of Transformers for Faster, Smaller and more Accurate NLP Models

Transformers have greatly advanced the state-of-the-art in Natural Language Processing (NLP) in recent years, but present very large computation and storage requirements. We observe that the design process of Transformers (pre-train a foundation model on a large dataset in a self-supervised manner, and subsequently fine-tune it for different downstream tasks) leads to task-specific models that are highly over-parameterized, adversely impacting both accuracy and inference efficiency. We propose AxFormer, a systematic framework that applies accuracy-driven approximations to create optimized transformer models for a given downstream task. AxFormer combines two key optimizations — accuracy-driven pruning and selective hard attention. Accuracy-driven pruning identifies and removes parts of the fine-tuned transformer that hinder performance on the given downstream task. Sparse hard-attention optimizes attention blocks in selected layers by eliminating irrelevant word aggregations, thereby helping the model focus only on the relevant parts of the input. In effect, AxFormer leads to models that are more accurate, while also being faster and smaller. Our experiments on GLUE and SQUAD tasks show that AxFormer models are up to 4.5% more accurate, while also being up to 2.5× faster and up to 3.2× smaller than conventional fine-tuned models. In addition, we demonstrate that AxFormer can be combined with previous efforts such as distillation or quantization to achieve further efficiency gains. Code is available at https://github.com/amrnag/Specialized-Transformers.
FvOR: Robust Joint Shape and Pose Optimization for Few-view Object Reconstruction

Reconstructing an accurate 3D object model from a few image observations remains a challenging problem in computer vision. State-of-the-art approaches typically assume accurate camera poses as input, which could be difficult to obtain in realistic settings. In this paper, we present FvOR, a learning-based object reconstruction method that predicts accurate 3D models given a few images with noisy input poses. The core of our approach is a fast and robust multi-view reconstruction algorithm to jointly refine 3D geometry and camera pose estimation using learnable neural network modules. We provide a thorough benchmark of state-of-the-art approaches for this problem on ShapeNet. Our approach achieves best-in-class results. It is also two orders of magnitude faster than the recent optimization-based approach IDR [67].
Exploiting Unlabeled Data with Vision and Language Models for Object Detection

Building robust and generic object detection frameworks requires scaling to larger label spaces and bigger training datasets. However, it is prohibitively costly to acquire annotations for thousands of categories at a large scale. We propose a novel method that leverages the rich semantics available in recent vision and language models to localize and classify objects in unlabeled images, effectively generating pseudo labels for object detection. Starting with a generic and class-agnostic region proposal mechanism, we use vision and language models to categorize each region of an image into any object category that is required for downstream tasks. We demonstrate the value of the generated pseudo labels in two specific tasks, open-vocabulary detection, where a model needs to generalize to unseen object categories, and semi-supervised object detection, where additional unlabeled images can be used to improve the model. Our empirical evaluation shows the effectiveness of the pseudo labels in both tasks, where we outperform competitive baselines and achieve a novel state-of-the-art for open-vocabulary object detection. Our code is available at https://github.com/xiaofeng94/VL-PLM .
Streaming vs. Functions: A Cost Perspective on Cloud Event Processing

In cloud event processing, data generated at the edge is processed in real-time by cloud resources. Both distributed stream processing (DSP) and Function-as-a-Service (FaaS) have been proposed to implement such event processing applications. FaaS emphasizes fast development and easy operation, while DSP emphasizes efficient handling of large data volumes. Despite their architectural differences, both can be used to model and implement loosely-coupled job graphs. In this paper, we consider the selection of FaaS and DSP from a cost perspective. We implement stateless and stateful workflows from the Theodolite benchmarking suite using cloud FaaS and DSP. In an extensive evaluation, we show how application type, cloud service provider, and runtime environment can influence the cost of application deployments and derive decision guidelines for cloud engineers.
India’s “Aadhaar” Biometric ID: Structure, Security, and Vulnerabilities

India's Aadhaar is the largest biometric identity system in history, designed to help deliver subsidies, benefits, and services to India's 1.4 billion residents. The Unique Identification Authority of India (UIDAI) is responsible for providing each resident (not each citizen) with a distinct identity—a 12-digit Aadhaar number—using their biometric and demographic details. We provide the first comprehensive description of the Aadhaar infrastructure, collating information across thousands of pages of public documents and releases, as well as direct discussions with Aadhaar developers. Critically, we describe the first known cryptographic issue within the system, and discuss how a workaround prevents it from being exploitable at scale. Further, we categorize and rate various security and privacy limitations and the corresponding threat actors, examine the legitimacy of alleged security breaches, and discuss improvements and mitigation strategies.
ContextBert: Enhanced Implicit Sentiment Analysis Using Implicit-sentiment-query Attention

In sentiment analysis, it is concerned recently that a significant portion of subjective sentences across different domains do not contain explicit sentiment words but still convey clear subjective sentiment, which is known as implicit sentiment. However, currently available methods do not perform well on implicit sentiment analysis. In most cases implicit sentiment can be inferred from the context. To address this issue, we propose the ContextBert model, which employs implicit-sentiment-query attention to find the contextual information related to the target sentence to enhance implicit sentiment analysis. Experimental results show that our method achieves state-of-the-art performance on SMP2019-ECISA and EmoContext-Implicit (reconstructed based on EmoContext, to enrich the implicit sentiment analysis dataset).
Portrait Eyeglasses and Shadow Removal by Leveraging 3D Synthetic Data

In portraits, eyeglasses may occlude facial regions and generate cast shadows on faces, which degrades the performance of many techniques like face verification and expression recognition. Portrait eyeglasses removal is critical in handling these problems. However, completely removing the eyeglasses is challenging because the lighting effects (e.g., cast shadows) caused by them are often complex. In this paper, we propose a novel framework to remove eyeglasses as well as their cast shadows from face images. The method works in a detect-then-remove manner, in which eyeglasses and cast shadows are both detected and then removed from images. Due to the lack of paired data for supervised training, we present a new synthetic portrait dataset with both intermediate and final supervisions for both the detection and removal tasks. Furthermore, we apply a cross-domain technique to fill the gap between the synthetic and real data. To the best of our knowledge, the proposed technique is the first to remove eyeglasses and their cast shadows simultaneously. The code and synthetic dataset are available at https://gethub.com/StoryMY/take-off-eyeglasses.
HM: Hybrid Masking for Few-Shot Segmentation

We study few-shot semantic segmentation that aims to segment a target object from a query image when provided with a few annotated support images of the target class. Several recent methods resort to a feature masking (FM) technique to discard irrelevant feature activations which eventually facilitates the reliable prediction of segmentation mask. A fundamental limitation of FM is the inability to preserve the fine-grained spatial details that affect the accuracy of segmentation mask, especially for small target objects. In this paper, we develop a simple, effective, and efficient approach to enhance feature masking (FM). We dub the enhanced FM as hybrid masking (HM). Specifically, we compensate for the loss of fine-grained spatial details in FM technique by investigating and leveraging a complementary basic input masking method. Experiments have been conducted on three publicly available benchmarks with strong few-shot segmentation (FSS) baselines. We empirically show improved performance against the current state-of-the-art methods by visible margins across different benchmarks. Our code and trained models are available at: https://github.com/moonsh/HM-Hybrid-Masking
How to Synthesize a Large-Scale and Trainable Micro-Expression Dataset?

This paper does not contain technical novelty but introduces our key discoveries in a data generation protocol, a database and insights. We aim to address the lack of large-scale datasets in micro-expression (MiE) recognition due to the prohibitive cost of data collection, which renders large-scale training less feasible. To this end, we develop a protocol to automatically synthesize large scale MiE training data that allow us to train improved recognition models for real-world test data. Specifically, we discover three types of Action Units (AUs) that can constitute trainable MiEs. These AUs come from real-world MiEs, early frames of macro-expression videos, and the relationship between AUs and expression categories defined by human expert knowledge. With these AUs, our protocol then employs large numbers of face images of various identities and an off-the-shelf face generator for MiE synthesis, yielding the MiE-X dataset. MiE recognition models are trained or pre-trained on MiE-X and evaluated on real-world test sets, where very competitive accuracy is obtained. Experimental results not only validate the effectiveness of the discovered AUs and MiE-X dataset but also reveal some interesting properties of MiEs: they generalize across faces, are close to early-stage macro-expressions, and can be manually defined. (This work was supported by the ARC Discovery Early Career Researcher Award (DE200101283) and the ARC Discovery Project (DP210102801).)
Online POI Recommendation: Learning Dynamic Geo-Human Interactions in Streams

Online POI recommendation strives to recommend future visitable places to users over time. It is crucial to improve the user experience of location-based social networking applications (e.g., Google Maps, Yelp). While numerous studies focus on capturing user visit preferences, geo-human interactions are ignored. However, users make visit decisions based on the status of geospatial contexts. In the meantime, such visits will change the status of geospatial contexts. Therefore, disregarding such geo-human interactions in streams will result in inferior recommendation performance. To fill this gap, in this article, we propose a novel deep interactive reinforcement learning framework to model geo-human interactions. Specifically, this framework has two main parts: the representation module and the imitation module. The purpose of the representation module is to capture geo-human interactions and convert them into embedding vectors (state). The imitation module is a reinforced agent whose job is to imitate user visit behavior by recommending next-visit POI (action) based on the state. Imitation performance is regarded as a reward signal to optimize the whole interactive framework. When the model converges, the imitation module can precisely perceive users and geospatial contexts to provide accurate POI recommendations. Finally, we conduct extensive experiments to validate the superiority of our framework.
XAI Sustainable Human in the Loop Maintenance

The field of Explainable Artificial Intelligence (XAI) is a relatively new approach to AI, with the aim to provide black box algorithms with human intelligible narrative functionality. It is most often in end-of-life considerations of the asset lifecycle that sustainability issues are encountered. Modern maintenance practice requires a holistic understanding of lifecycle and options for sustainable asset treatments. human in the loop solutions offer a way to leverage both machine and human skill sets to provide the next level of automaton solutions for industrial maintenance activities. This paper presents a framework for human in the loop Intelligent and Sustainable Maintenance. In bridging the gap between machines and humans XAI leverages the best of both worlds to provide a new level of agility to cyber assisted maintenance activities and full lifecycle consideration of assets; a notion that is necessary throughout the organization in the achievement of sustainability goals set by governments around the world in the achievement of a net zero carbon emission economy.