Reviewerly/exHarmony huggingface dataset:
Waypoint Models for Instruction-guided Navigation in Continuous Environments

Little inquiry has explicitly addressed the role of action spaces in language-guided visual navigation – either in terms of its effect on navigation success or the efficiency with which a robotic agent could execute the resulting trajectory. Building on the recently released VLN-CE [24] setting for instruction following in continuous environments, we develop a class of language-conditioned waypoint prediction networks to examine this question. We vary the expressivity of these models to explore a spectrum between low-level actions and continuous waypoint prediction. We measure task performance and estimated execution time on a profiled LoCoBot [1] robot. We find more expressive models result in simpler, faster to execute trajectories, but lower-level actions can achieve better navigation metrics by approximating shortest paths better. Further, our models outperform prior work in VLN-CE and set a new state-of-the-art on the public leaderboard – increasing success rate by 4% with our best model on this challenging task.
Mask is All You Need

Due to the large success in object detection and instance segmentation, Mask R-CNN attracts great attention and is widely adopted as a strong baseline for arbitrary-shaped scene text detection and spotting. However, two issues remain to be settled. The first is dense text case, which is easy to be neglected but quite practical. There may exist multiple instances in one proposal, which makes it difficult for the mask head to distinguish different instances and degrades the performance. In this work, we argue that the performance degradation results from the learning confusion issue in the mask head. We propose to use an MLP decoder instead of the "deconv-conv" decoder in the mask head, which alleviates the issue and promotes robustness significantly. And we propose instance-aware mask learning in which the mask head learns to predict the shape of the whole instance rather than classify each pixel to text or non-text. With instance-aware mask learning, the mask branch can learn separated and compact masks. The second is that due to large variations in scale and aspect ratio, RPN needs complicated anchor settings, making it hard to maintain and transfer across different datasets. To settle this issue, we propose an adaptive label assignment in which all instances especially those with extreme aspect ratios are guaranteed to be associated with enough anchors. Equipped with these components, the proposed method named MAYOR achieves state-of-the-art performance on five benchmarks including DAST1500, MSRA-TD500, ICDAR2015, CTW1500, and Total-Text.
iText: Hands-free Text Entry on an Imaginary Keyboard for Augmented Reality Systems

Text entry is an important and frequent task in interactive devices including augmented reality head-mounted displays (AR HMDs). In current AR HMDs, there are still two main open challenges to overcome for efficient and usable text entry: arm fatigue due to mid-air input and visual occlusion because of their small see-through displays. To address these challenges, we present iText, a technique for AR HMDs that is hands-free and is based on an imaginary (invisible) keyboard. We first show that it is feasible and practical to use an imaginary keyboard on AR HMDs. Then, we evaluated its performance and usability with three hands-free selection mechanisms: eye blinks (E-Type), dwell (D-Type), and swipe gestures (G-Type). Our results show that users could achieve an average text entry speed of 11.95, 9.03 and 9.84 words per minutes (WPM) with E-Type, D-Type, and G-Type, respectively. Given that iText with E-Type outperformed the other two selection mechanisms in text entry rate and subjective feedback, we ran a third, 5-day study. Our results show that iText with E-Type can achieve an average text entry rate of 13.76 WPM with a mean word error rate of 1.5%. In short, iText can enable efficient eyes-free text entry and can be useful for various application scenarios in AR HMDs.
Don't Disturb Me: Challenges of Interacting with Software Bots on Open Source Software Projects

Software bots are used to streamline tasks in Open Source Software (OSS) projects' pull requests, saving development cost, time, and effort. However, their presence can be disruptive to the community. We identified several challenges caused by bots in pull request interactions by interviewing 21 practitioners, including project maintainers, contributors, and bot developers. In particular, our findings indicate noise as a recurrent and central problem. Noise affects both human communication and development workflow by overwhelming and distracting developers. Our main contribution is a theory of how human developers perceive annoying bot behaviors as noise on social coding platforms. This contribution may help practitioners understand the effects of adopting a bot, and researchers and tool designers may leverage our results to better support human-bot interaction on social coding platforms.
Teleportation in Virtual Reality; A Mini-Review

Teleportation is a widely implemented virtual locomotion technique that allows users to navigate beyond the confines of available tracking space with a low possibility of inducing virtual reality (VR) sickness. This paper provides a comprehensive overview of prior research on teleportation. We report results from user studies that have evaluated teleportation in comparison to other locomotion methods and survey improved versions of teleportation. We identify a number of areas for future research.
Cybersecurity Awareness Framework for Academia

Cybersecurity is a multifaceted global phenomenon representing complex socio-technical challenges for governments and private sectors. With technology constantly evolving, the types and numbers of cyberattacks affect different users in different ways. The majority of recorded cyberattacks can be traced to human errors. Despite being both knowledge- and environment-dependent, studies show that increasing users’ cybersecurity awareness is found to be one of the most effective protective approaches. However, the intangible nature, socio-technical dependencies, constant technological evolutions, and ambiguous impact make it challenging to offer comprehensive strategies for better communicating and combatting cyberattacks. Research in the industrial sector focused on creating institutional proprietary risk-aware cultures. In contrast, in academia, where cybersecurity awareness should be at the core of an academic institution’s mission to ensure all graduates are equipped with the skills to combat cyberattacks, most of the research focused on understanding students’ attitudes and behaviors after infusing cybersecurity awareness topics into some courses in a program. This work proposes a conceptual Cybersecurity Awareness Framework to guide the implementation of systems to improve the cybersecurity awareness of graduates in any academic institution. This framework comprises constituents designed to continuously improve the development, integration, delivery, and assessment of cybersecurity knowledge into the curriculum of a university across different disciplines and majors; this framework would thus lead to a better awareness among all university graduates, the future workforce. This framework may be adjusted to serve as a blueprint that, once adjusted by academic institutions to accommodate their missions, guides institutions in developing or amending their policies and procedures for the design and assessment of cybersecurity awareness.
An Empirical Analysis of Privacy in the Lightning Network

Payment channel networks, and the Lightning Network in particular, seem to offer a solution to the lack of scalability and privacy offered by Bitcoin and other blockchain-based cryptocurrencies. Previous research has focused on the scalability, availability, and crypto-economics of the Lightning Network, but relatively little attention has been paid to exploring the level of privacy it achieves in practice. This paper presents a thorough analysis of the privacy offered by the Lightning Network, by presenting several attacks that exploit publicly available information about the network in order to learn information that is designed to be kept secret, such as how many coins a node has available or who the sender and recipient are in a payment routed through the network.
Construction of EBRB classifier for imbalanced data based on Fuzzy C-Means clustering

The Extended Belief Rule-Based (EBRB) system has been widely used to solve the real-world problems concerning with incompleteness, uncertainty, and ambiguity. However, EBRB is essentially a data-driven method, in which each rule is obtained from training data. Therefore, the generated extended belief rules may be severely biased when dealing with data with imbalanced classes. In this case, the number of the rules generated by the samples of majority classes (i.e., negative samples) may be much larger than those of minority classes (i.e., positive samples). Thus, the class imbalance may lead to significant biases in system decision-making. In order to resolve this problem, this paper proposes a novel EBRB system based on fuzzy C-means clustering (FCM-EBRB). First, we adopt FCM clustering to oversample the positive samples and undersample the negative ones, so as to achieve the balance between them. Next, this paper improves the construction method of EBRB and optimizes the system through an efficient parameter learning strategy. Finally, this paper conducts comprehensive comparison experiments on a binary classification synthetic dataset and 11 commonly used KEEL public class imbalance datasets. Experimental results show that the proposed method can effectively reduce the scale of the rule base and achieve high inference accuracy, especially for imbalanced data.
SigNet: A Novel Deep Learning Framework for Radio Signal Classification

Deep learning methods achieve great success in many areas due to their powerful feature extraction capabilities and end-to-end training mechanism, and recently they are also introduced for radio signal modulation classification. In this paper, we propose a novel deep learning framework called SigNet, where a signal-to-matrix (S2M) operator is adopted to convert the original signal into a square matrix first and is co-trained with a follow-up CNN architecture for classification. This model is further accelerated by integrating 1D convolution operators, leading to the upgraded model SigNet2.0. The simulations on two signal datasets show that both SigNet and SigNet2.0 outperform a number of well-known baselines. More interestingly, our proposed models behave extremely well in small-sample learning when only a small training dataset is provided. They can achieve a relatively high accuracy even when 1% training data are kept, while other baseline models may lose their effectiveness much more quickly as the datasets get smaller. Such result suggests that SigNet/SigNet2.0 could be extremely useful in the situations where labeled signal data are difficult to obtain. The visualization of the output features of our models demonstrates that our model can well divide different modulation types of signals in the feature hyper-space.
Rethinking Modern Communication from Semantic Coding to Semantic Communication

Modern communications are usually designed to pursue a higher bit-level precision and fewer bits while transmitting a message. This article rethinks these two major features and introduces the concept and advantage of semantics that characterizes a new kind of semantics-aware communication framework, incorporating both the semantic encoding and the semantic communication problem. After analyzing the underlying defects of existing semantics-aware techniques, we establish a confidence-based distillation mechanism for the joint semantics-noise coding (JSNC) problem and a reinforcement learning (RL)-powered semantic communication paradigm that endows a system the ability to convey the semantics instead of pursuing the bit level accuracy. On top of these technical contributions, this work provides a new insight to understand how the semantics are processed and represented in a semantics-aware coding and communication system, and verifies the significant benefits of doing so. Targeted on the next generation's semantics-aware communication, some critical concerns and open challenges such as the information overhead, semantic security and implementation cost are also discussed and envisioned.
A survey of methods, datasets and evaluation metrics for visual question answering

Visual Question Answering (VQA) is a multi-disciplinary research problem that has captured the attention of both computer vision as well as natural language processing researchers. In Visual Question Answering, a system is given an image; a question in a natural language related to that image as an input, and the VQA system is required to give an answer in natural language as an output. A VQA algorithm may require common sense reasoning over the information contained in the image and world knowledge to produce the right answer. In this paper, we have discussed some of the core concepts used in VQA systems and present a comprehensive survey of efforts in the past to address this problem. Apart from traditional VQA models, we have also discussed visual question answering models that require reading texts present in images and evaluated on recently developed datasets like TextVQA, ST-VQA, and OCR-VQA. Apart from standard datasets discussed in previous surveys, we have also discussed some new datasets developed in 2019 and 2020 such as GQA, OK-VQA, TextVQA, ST-VQA, and OCR-VQA. The new evaluation metrics such as BLEU, MPT, METEOR, Average Normalized Levenshtein Similarity (ANLS), Validity, Plausibility, Distribution, Consistency, Grounding, F1-Score are explained together with the evaluation metrics discussed by previous surveys. We conclude our survey with a discussion on open issues in each phase of the VQA task and present some promising future directions.
Graph Convolutional Multi-modal Hashing for Flexible Multimedia Retrieval

Multi-modal hashing makes an important contribution to multimedia retrieval, where a key challenge is to encode heterogeneous modalities into compact hash codes. To solve this dilemma, graph-based multi-modal hashing methods generally define individual affinity matrix of each independent modality and apply linear algorithm for heterogeneous modalities fusion and compact hash learning. Several other methods construct graph Laplacian matrix based on semantic information to help learn discriminative hash code. However, these conventional methods roughly ignore the structural similarity of training set and the complex relations among multi-modal samples, which leads to unsatisfactory complementarity of fused hash codes. More notably, they are faced with two other important problems: huge computing and storage costs caused by graph construction and partial modality feature lost problem when incomplete query sample comes. In this paper, we propose a Flexible Graph Convolutional Multi-modal Hashing (FGCMH) method that adopts GCNs with linear complexity to preserve both the modality-individual and modality-fused structural similarity for discriminative hash learning. Necessarily, accurate multimedia retrieval can be performed on complete and incomplete datasets with our method. Specifically, multiple modality-individual GCNs under semantic guidance are proposed to act on each individual modality independently for intra-modality similarity preserving, then the output representations are fused into a fusion graph with adaptive weighting scheme. Hash GCN and semantic GCN, which share parameters in the first two layers, propagate fusion information and generate hash codes under high-level label space supervision. In the query stage, our method adaptively captures various multi-modal contents in a flexible and robust way, even if partial modality features are lost. Experimental results on three publicly datasets show the flexibility and effectiveness of our proposed method.
A model selection approach to structural equation modelling: A critical evaluation and a road map for ecologists

Abstract Structural equation modelling (SEM) can illuminate complex interaction networks of the sort found in ecology. However, selecting optimally complex, data‐supported SEM models and quantifying their uncertainty are difficult processes. To this end, we recommend a formal model selection approach (MSA) that uses information criteria. Using a suite of numerical simulations, we compare MSA‐SEM against two traditional methods. We find that MSA‐SEM exhibits superior, unbiased results under the suboptimal realistic conditions characteristic of ecological studies. We then provide a road map for MSA‐SEM and demonstrate its use via a case study. We illustrate the unique abilities of SEM to confirm a network structure within the realm of known causal pathways and delineate the boundaries within which MSA‐SEM should be applied.
MLA: A New Mutated Leader Algorithm for Solving Optimization Problems

Optimization plays an effective role in various disciplines of science and engineering. Optimization problems should either be optimized using the appropriate method (i.e., minimization or maximization). Optimization algorithms are one of the efficient and effective methods in providing quasi-optimal solutions for these type of problems. In this study, a new algorithm called the Mutated Leader Algorithm (MLA) is presented. The main idea in the proposed MLA is to update the members of the algorithm population in the search space based on the guidance of a mutated leader. In addition to information about the best member of the population, the mutated leader also contains information about the worst member of the population, as well as other normal members of the population. The proposed MLA is mathematically modeled for implementation on optimization problems. A standard set consisting of twenty-three objective functions of different types of unimodal, fixed-dimensional multimodal, and high-dimensional multimodal is used to evaluate the ability of the proposed algorithm in optimization. Also, the results obtained from the MLA are compared with eight well-known algorithms. The results of optimization of objective functions show that the proposed MLA has a high ability to solve various optimization problems. Also, the analysis and comparison of the performance of the proposed MLA against the eight compared algorithms indicates the superiority of the proposed algorithm and ability to provide more suitable quasi-optimal solutions.
DehazeFlow: Multi-scale Conditional Flow Network for Single Image Dehazing

Single image dehazing is a crucial and preliminary task for many computer vision applications, making progress with deep learning. The dehazing task is an ill-posed problem since the haze in the image leads to the loss of information. Thus, there are multiple feasible solutions for image restoration of a hazy image. Most existing methods learn a deterministic one-to-one mapping between a hazy image and its ground-truth, which ignores the ill-posedness of the dehazing task. To solve this problem, we propose DehazeFlow, a novel single image dehazing framework based on conditional normalizing flow. Our method learns the conditional distribution of haze-free images given a hazy image, enabling the model to sample multiple dehazed results. Furthermore, we propose an attention-based coupling layer to enhance the expression ability of a single flow step, which converts natural images into latent space and fuses features of paired data. These designs enable our model to achieve state-of-the-art performance while considering the ill-posedness of the task. We carry out sufficient experiments on both synthetic datasets and real-world hazy images to illustrate the effectiveness of our method. The extensive experiments indicate that DehazeFlow surpasses the state-of-the-art methods in terms of PSNR, SSIM, LPIPS, and subjective visual effects.
GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications

Freehand gesture is an essential input modality for modern Augmented Reality (AR) user experiences. However, developing AR applications with customized hand interactions remains a challenge for end-users. Therefore, we propose GesturAR, an end-to-end authoring tool that supports users to create in-situ freehand AR applications through embodied demonstration and visual programming. During authoring, users can intuitively demonstrate the customized gesture inputs while referring to the spatial and temporal context. Based on the taxonomy of gestures in AR, we proposed a hand interaction model which maps the gesture inputs to the reactions of the AR contents. Thus, users can author comprehensive freehand applications using trigger-action visual programming and instantly experience the results in AR. Further, we demonstrate multiple application scenarios enabled by GesturAR, such as interactive virtual objects, robots, and avatars, room-level interactive AR spaces, embodied AR presentations, etc. Finally, we evaluate the performance and usability of GesturAR through a user study.
NFT based Fundraising System for Preserving Cultural Heritage: Heirloom

Cultural heritage assets are in danger of extinction or damage due to lack of publicity and financial problems. Technological advances can play a role in their preservation and promotion. This study aims to create a blockchain-based cultural property protection system which we named the Heirloom. The proposed system uses blockchain and IPFS. This system will allow foundations to receive funding to protect cultural assets without using an intermediary. The cultural assets are transformed into unique digital items using the NFT (Non-Fungible Token) technology. The metadata of the created NFTs is stored in the distributed file system IPFS (InterPlanetary File System). An autonomous working system is provided with smart contracts. The supporters give donations to earn their share of protection and maintenance rights. The proof of concept implementation is promising. A case study on protecting old olive trees in Milas has also started with a local foundation. Possible outcomes will be the ease of getting funds for preserving cultural heritage and increasing awareness. Future studies will include working on different methods for decreasing the costs of the system and integrating augmented and virtual reality technologies.
A Taxonomy of Interaction Techniques for Immersive Augmented Reality based on an Iterative Literature Review

Developers of interactive systems have a variety of interaction techniques to choose from, each with individual strengths and limitations in terms of the considered task, context, and users. While there are taxonomies for desktop, mobile, and virtual reality applications, augmented reality (AR) taxonomies have not been established yet. However, recent advances in immersive AR technology (i.e., head-worn or projection-based AR), such as the emergence of untethered headsets with integrated gesture and speech sensors, have enabled the inclusion of additional input modalities and, therefore, novel multimodal interaction methods have been introduced. To provide an overview of interaction techniques for current immersive AR systems, we conducted a literature review of publications between 2016 and 2021. Based on 44 relevant papers, we developed a comprehensive taxonomy focusing on two identified dimensions – task and modality. We further present an adaptation of an iterative taxonomy development method to the field of human-computer interaction. Finally, we discuss observed trends and implications for future work.