OpenRaiser/Intern-Atlas huggingface dataset:
High-Resolution Depth Estimation for 360-degree Panoramas through Perspective and Panoramic Depth Images Registration
Venue: arXiv cs.CV 2022
Year: 2022

We propose a novel approach to compute high-resolution (2048x1024 and higher) depths for panoramas that is significantly faster and qualitatively and qualitatively more accurate than the current state-of-the-art method (360MonoDepth). As traditional neural network-based methods have limitations in the output image sizes (up to 1024x512) due to GPU memory constraints, both 360MonoDepth and our method rely on stitching multiple perspective disparity or depth images to come out a unified panoramic depth map. However, to achieve globally consistent stitching, 360MonoDepth relied on solving extensive disparity map alignment and Poisson-based blending problems, leading to high computation time. Instead, we propose to use an existing panoramic depth map (computed in real-time by any panorama-based method) as the common target for the individual perspective depth maps to register to. This key idea made producing globally consistent stitching results from a straightforward task. Our experiments show that our method generates qualitatively better results than existing panorama-based methods, and further outperforms them quantitatively on datasets unseen by these methods.
A New Communication Protocol with Self Error Correction
Venue: arXiv cs.NI 2022
Year: 2022

Communication in poor network environment is always a difficult problem, since troubles such as bit errors and packet loss may often occur. It is generally believed that it is impossible to transmit data both accurately and efficiently in this case. However, this paper provides a method to transmit data efficiently on the line where bit error may occur by utilizing Hamming code principle. If the sender adds a small amount of redundant data to the data to be sent, the receiver can self-correct them when an error is detected. This approach takes advantage of the value of packets with errors, which should have been discarded, reduce the number of re-transmissions and improve transmission efficiency. Based on this method, this paper designs a custom protocol which works in the data link layer and network layer. Finally, this paper verifies the protocol through mathematical simulation.
Visual SLAM: What are the Current Trends and What to Expect?
Venue: arXiv cs.CV 2022
Year: 2022

Vision-based sensors have shown significant performance, accuracy, and efficiency gain in Simultaneous Localization and Mapping (SLAM) systems in recent years. In this regard, Visual Simultaneous Localization and Mapping (VSLAM) methods refer to the SLAM approaches that employ cameras for pose estimation and map generation. We can see many research works that demonstrated VSLAMs can outperform traditional methods, which rely only on a particular sensor, such as a Lidar, even with lower costs. VSLAM approaches utilize different camera types (e.g., monocular, stereo, and RGB-D), have been tested on various datasets (e.g., KITTI, TUM RGB-D, and EuRoC) and in dissimilar environments (e.g., indoors and outdoors), and employ multiple algorithms and methodologies to have a better understanding of the environment. The mentioned variations have made this topic popular for researchers and resulted in a wide range of VSLAMs methodologies. In this regard, the primary intent of this survey is to present the recent advances in VSLAM systems, along with discussing the existing challenges and trends. We have given an in-depth literature survey of forty-five impactful papers published in the domain of VSLAMs. We have classified these manuscripts by different characteristics, including the novelty domain, objectives, employed algorithms, and semantic level. We also discuss the current trends and future directions that may help researchers investigate them.
On Bisimulation in Absence of Restriction
Venue: arXiv cs.LO 2022
Year: 2022

We revisit the standard bisimulation equalities in process models free of the restriction operator. As is well-known, in general the weak bisimilarity is coarser than the strong bisimilarity because it abstracts from internal actions. In absence of restriction, those internal actions become somewhat visible, so one might wonder if the weak bisimilarity is still 'weak'. We show that in both CCScore (i.e., Milner's standard CCS without $τ$-prefix, summation and relabelling) and its higher-order variant (named HOCCScore), the weak bisimilarity indeed remains weak, i.e., still strictly coarser than the strong bisimilarity, even without the restriction operator. These results can be extended to other first-order or higher-order process models. Essentially, this is due to the direct or indirect existence of the replication operation, which can keep a process retaining its state (i.e., capacity of interaction). By virtue of these observations, we examine a variant of the weak bisimilarity, called quasi-strong bisimilarity. This quasi-strong bisimilarity requires the matching of internal actions to be conducted in the strong manner, as for the strong bisimilarity, and the matching of visible actions to have no trailing internal actions. We exhibit that in CCScore without the restriction operator, the weak bisimilarity exactly collapses onto this quasi-strong bisimilarity, which is moreover shown to coincide with the branching bisimilarity. These results reveal that in absence of the restriction operation, some ingredient of the weak bisimilarity indeed turns into strong, particularly the matching of internal actions.
Motion-Based Weak Supervision for Video Parsing with Application to Colonoscopy
Venue: arXiv cs.CV 2022
Year: 2022

We propose a two-stage unsupervised approach for parsing videos into phases. We use motion cues to divide the video into coarse segments. Noisy segment labels are then used to weakly supervise an appearance-based classifier. We show the effectiveness of the method for phase detection in colonoscopy videos.
NGEP: A Graph-based Event Planning Framework for Story Generation
Venue: arXiv cs.CL 2022
Year: 2022

To improve the performance of long text generation, recent studies have leveraged automatically planned event structures (i.e. storylines) to guide story generation. Such prior works mostly employ end-to-end neural generation models to predict event sequences for a story. However, such generation models struggle to guarantee the narrative coherence of separate events due to the hallucination problem, and additionally the generated event sequences are often hard to control due to the end-to-end nature of the models. To address these challenges, we propose NGEP, an novel event planning framework which generates an event sequence by performing inference on an automatically constructed event graph and enhances generalisation ability through a neural event advisor. We conduct a range of experiments on multiple criteria, and the results demonstrate that our graph-based neural framework outperforms the state-of-the-art (SOTA) event planning approaches, considering both the performance of event sequence generation and the effectiveness on the downstream task of story generation.
RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses
Venue: arXiv cs.IR 2022
Year: 2022

Recently, substantial progress has been made in text ranking based on pretrained language models such as BERT. However, there are limited studies on how to leverage more powerful sequence-to-sequence models such as T5. Existing attempts usually formulate text ranking as classification and rely on postprocessing to obtain a ranked list. In this paper, we propose RankT5 and study two T5-based ranking model structures, an encoder-decoder and an encoder-only one, so that they not only can directly output ranking scores for each query-document pair, but also can be fine-tuned with "pairwise" or "listwise" ranking losses to optimize ranking performances. Our experiments show that the proposed models with ranking losses can achieve substantial ranking performance gains on different public text ranking data sets. Moreover, when fine-tuned with listwise ranking losses, the ranking model appears to have better zero-shot ranking performance on out-of-domain data sets compared to the model fine-tuned with classification losses.
Digital Asset Valuation: A Study on Domain Names, Email Addresses, and NFTs
Venue: arXiv cs.IR 2022
Year: 2022

Existing works on valuing digital assets on the Internet typically focus on a single asset class. To promote the development of automated valuation techniques, preferably those that are generally applicable to multiple asset classes, we construct DASH, the first Digital Asset Sales History dataset that features multiple digital asset classes spanning from classical to blockchain-based ones. Consisting of 280K transactions of domain names (DASH_DN), email addresses (DASH_EA), and non-fungible token (NFT)-based identifiers (DASH_NFT), such as Ethereum Name Service names, DASH advances the field in several aspects: the subsets DASH_DN, DASH_EA, and DASH_NFT are the largest freely accessible domain name transaction dataset, the only publicly available email address transaction dataset, and the first NFT transaction dataset that focuses on identifiers, respectively. We build strong conventional feature-based models as the baselines for DASH. We next explore deep learning models based on fine-tuning pre-trained language models, which have not yet been explored for digital asset valuation in the previous literature. We find that the vanilla fine-tuned model already performs reasonably well, outperforming all but the best-performing baselines. We further propose improvements to make the model more aware of the time sensitivity of transactions and the popularity of assets. Experimental results show that our improved model consistently outperforms all the other models across all asset classes on DASH.
Preliminary Analysis of Channel Capacity in Air to ground LoS MIMO Communication Based on A Cloud Modeling Method
Venue: arXiv cs.IT 2022
Year: 2022

Since the orthogonality of the line-of-sight multiple input multiple output (LoS MIMO) channel is only available within the Rayleigh distance, coverage of communication systems is restricted due to the finite implementation spacing of antennas. However, media with different permittivity in the transmission path are likely to loosen the requirement for antenna spacing. Such a conclusion could be enlightening in an air-to-ground LoS MIMO scenario considering the existence of clouds in the troposphere. To analyze the random phase variations in the presence of a single-layer cloud, we propose and modify a new cloud modeling method fit for LoS MIMO scene based on real-measurement data. Then, the preliminary analysis of channel capacity is conducted based on the simulation result.
POGD: Gradient Descent with New Stochastic Rules
Venue: arXiv cs.LG 2022
Year: 2022

There introduce Particle Optimized Gradient Descent (POGD), an algorithm based on the gradient descent but integrates the particle swarm optimization (PSO) principle to achieve the iteration. From the experiments, this algorithm has adaptive learning ability. The experiments in this paper mainly focus on the training speed to reach the target value and the ability to prevent the local minimum. The experiments in this paper are achieved by the convolutional neural network (CNN) image classification on the MNIST and cifar-10 datasets.
Analysis of Master Vein Attacks on Finger Vein Recognition Systems
Venue: arXiv cs.CV 2022
Year: 2022

Finger vein recognition (FVR) systems have been commercially used, especially in ATMs, for customer verification. Thus, it is essential to measure their robustness against various attack methods, especially when a hand-crafted FVR system is used without any countermeasure methods. In this paper, we are the first in the literature to introduce master vein attacks in which we craft a vein-looking image so that it can falsely match with as many identities as possible by the FVR systems. We present two methods for generating master veins for use in attacking these systems. The first uses an adaptation of the latent variable evolution algorithm with a proposed generative model (a multi-stage combination of beta-VAE and WGAN-GP models). The second uses an adversarial machine learning attack method to attack a strong surrogate CNN-based recognition system. The two methods can be easily combined to boost their attack ability. Experimental results demonstrated that the proposed methods alone and together achieved false acceptance rates up to 73.29% and 88.79%, respectively, against Miura's hand-crafted FVR system. We also point out that Miura's system is easily compromised by non-vein-looking samples generated by a WGAN-GP model with false acceptance rates up to 94.21%. The results raise the alarm about the robustness of such systems and suggest that master vein attacks should be considered an important security measure.
Unboxing Trustworthiness through Quantum Internet
Venue: arXiv cs.NI 2022
Year: 2022

The broad adoption of the Internet of Things during the last decade has widened the application horizons of distributed sensor networks, ranging from smart home appliances to automation, including remote sensing. Typically, these distributed systems are composed of several nodes attached to sensing devices linked by a heterogeneous communication network. The unreliable nature of these systems (e.g., devices might run out of energy or communications might become unavailable) drives practitioners to implement heavyweight fault tolerance mechanisms to identify those untrustworthy nodes that are misbehaving erratically and, thus, ensure that the sensed data from the IoT domain are correct. The overhead in the communication network degrades the overall system, especially in scenarios with limited available bandwidth that are exposed to severely harsh conditions. Quantum Internet might be a promising alternative to minimize traffic congestion and avoid worsening reliability due to the link saturation effect by using a quantum consensus layer. In this regard, the purpose of this paper is to explore and simulate the usage of quantum consensus architecture in one of the most challenging natural environments in the world where researchers need a responsive sensor network: the remote sensing of permafrost in Antarctica. More specifically, this paper 1) describes the use case of permafrost remote sensing in Antarctica, 2) proposes the usage of a quantum consensus management plane to reduce the traffic overhead associated with fault tolerance protocols, and 3) discusses, by means of simulation, possible improvements to increase the trustworthiness of a holistic telemetry system by exploiting the complexity reduction offered by the quantum parallelism. Collected insights from this research can be generalized to current and forthcoming IoT environments.
End-to-End Integration of Speech Recognition, Dereverberation, Beamforming, and Self-Supervised Learning Representation
Venue: arXiv cs.SD 2022
Year: 2022

Self-supervised learning representation (SSLR) has demonstrated its significant effectiveness in automatic speech recognition (ASR), mainly with clean speech. Recent work pointed out the strength of integrating SSLR with single-channel speech enhancement for ASR in noisy environments. This paper further advances this integration by dealing with multi-channel input. We propose a novel end-to-end architecture by integrating dereverberation, beamforming, SSLR, and ASR within a single neural network. Our system achieves the best performance reported in the literature on the CHiME-4 6-channel track with a word error rate (WER) of 1.77%. While the WavLM-based strong SSLR demonstrates promising results by itself, the end-to-end integration with the weighted power minimization distortionless response beamformer, which simultaneously performs dereverberation and denoising, improves WER significantly. Its effectiveness is also validated on the REVERB dataset.
Scaling Laws for Reward Model Overoptimization
Venue: arXiv cs.LG 2022
Year: 2022

In reinforcement learning from human feedback, it is common to optimize against a reward model trained to predict human preferences. Because the reward model is an imperfect proxy, optimizing its value too much can hinder ground truth performance, in accordance with Goodhart's law. This effect has been frequently observed, but not carefully measured due to the expense of collecting human preference data. In this work, we use a synthetic setup in which a fixed "gold-standard" reward model plays the role of humans, providing labels used to train a proxy reward model. We study how the gold reward model score changes as we optimize against the proxy reward model using either reinforcement learning or best-of-$n$ sampling. We find that this relationship follows a different functional form depending on the method of optimization, and that in both cases its coefficients scale smoothly with the number of reward model parameters. We also study the effect on this relationship of the size of the reward model dataset, the number of reward model and policy parameters, and the coefficient of the KL penalty added to the reward in the reinforcement learning setup. We explore the implications of these empirical results for theoretical considerations in AI alignment.
Self-Supervised Representation Learning for CAD
Venue: arXiv cs.CV 2022
Year: 2022

The design of man-made objects is dominated by computer aided design (CAD) tools. Assisting design with data-driven machine learning methods is hampered by lack of labeled data in CAD's native format; the parametric boundary representation (B-Rep). Several data sets of mechanical parts in B-Rep format have recently been released for machine learning research. However, large scale databases are largely unlabeled, and labeled datasets are small. Additionally, task specific label sets are rare, and costly to annotate. This work proposes to leverage unlabeled CAD geometry on supervised learning tasks. We learn a novel, hybrid implicit/explicit surface representation for B-Rep geometry, and show that this pre-training significantly improves few-shot learning performance and also achieves state-of-the-art performance on several existing B-Rep benchmarks.
RCareWorld: A Human-centric Simulation World for Caregiving Robots
Venue: arXiv cs.RO 2022
Year: 2022

We present RCareWorld, a human-centric simulation world for physical and social robotic caregiving designed with inputs from stakeholders, including care recipients, caregivers, occupational therapists, and roboticists. RCareWorld has realistic human models of care recipients with mobility limitations and caregivers, home environments with multiple levels of accessibility and assistive devices, and robots commonly used for caregiving. It interfaces with various physics engines to model diverse material types necessary for simulating caregiving scenarios, and provides the capability to plan, control, and learn both human and robot control policies by integrating with state-of-the-art external planning and learning libraries, and VR devices. We propose a set of realistic caregiving tasks in RCareWorld as a benchmark for physical robotic caregiving and provide baseline control policies for them. We illustrate the high-fidelity simulation capabilities of RCareWorld by demonstrating the execution of a policy learnt in simulation for one of these tasks on a real-world setup. Additionally, we perform a real-world social robotic caregiving experiment using behaviors modeled in RCareWorld. Robotic caregiving, though potentially impactful towards enhancing the quality of life of care recipients and caregivers, is a field with many barriers to entry due to its interdisciplinary facets. RCareWorld takes the first step towards building a realistic simulation world for robotic caregiving that would enable researchers worldwide to contribute to this impactful field. Demo videos and supplementary materials can be found at: https://emprise.cs.cornell.edu/rcareworld/.
Supervised Contrastive Learning with Tree-Structured Parzen Estimator Bayesian Optimization for Imbalanced Tabular Data
Venue: arXiv cs.LG 2022
Year: 2022

Class imbalance has a detrimental effect on the predictive performance of most supervised learning algorithms as the imbalanced distribution can lead to a bias preferring the majority class. To solve this problem, we propose a Supervised Contrastive Learning (SCL) method with Tree-structured Parzen Estimator (TPE) technique for imbalanced tabular datasets. Contrastive learning (CL) can extract the information hidden in data even without labels and has shown some potential for imbalanced learning tasks. SCL further considers the label information based on CL, which also addresses the insufficient data augmentation techniques of tabular data. Therefore, in this work, we propose to use SCL to learn a discriminative representation of imbalanced tabular data. Additionally, the hyper-parameter temperature of SCL has a decisive influence on the performance and is difficult to tune. We introduce TPE, a well-known Bayesian optimization technique, to automatically select the best temperature. Experiments are conducted on both binary and multi-class imbalanced tabular datasets. As shown in the results obtained, TPE outperforms three other hyper-parameter optimization (HPO) methods such as grid search, random search, and genetic algorithm. More importantly, the proposed SCL-TPE method achieves much-improved performance compared with the state-of-the-art methods.
Sparse arrays in R: the spray package
Venue: arXiv cs.SC 2022
Year: 2022

In this short article I introduce the spray package, which provides some functionality for handling sparse arrays. The package uses the C++ Standard Template Library's map class to store and retrieve elements. One natural application for sparse arrays is multivariate polynomials and I give two examples of the package in use, one drawn from the fields of random walks on lattices and one from the field of recreational combinatorics. The package is available on CRAN at https://CRAN.R-project.org/package=spray.
Enhancing Out-of-Distribution Detection in Natural Language Understanding via Implicit Layer Ensemble
Venue: arXiv cs.CL 2022
Year: 2022

Out-of-distribution (OOD) detection aims to discern outliers from the intended data distribution, which is crucial to maintaining high reliability and a good user experience. Most recent studies in OOD detection utilize the information from a single representation that resides in the penultimate layer to determine whether the input is anomalous or not. Although such a method is straightforward, the potential of diverse information in the intermediate layers is overlooked. In this paper, we propose a novel framework based on contrastive learning that encourages intermediate features to learn layer-specialized representations and assembles them implicitly into a single representation to absorb rich information in the pre-trained language model. Extensive experiments in various intent classification and OOD datasets demonstrate that our approach is significantly more effective than other works.