OpenRaiser/Intern-Atlas huggingface dataset:
Metaverse: Survey, Applications, Security, and Opportunities
Venue: arXiv cs.DB 2022
Year: 2022

As a fusion of various emerging digital technologies, the Metaverse aims to build a virtual shared digital space. It is closely related to extended reality, digital twin, blockchain, and other technologies. Its goal is to build a digital space based on the real world, form a virtual economic system, and expand the space of human activities, which injects new vitality into the social, economic, and other fields. In this article, we make the following contributions. We first introduce the basic concepts such as the development process, definition, and characteristics of the Metaverse. After that, we analyze the overall framework and supporting technologies of the Metaverse and summarize the status of common fields. Finally, we focus on the security and privacy issues that exist in the Metaverse, and give corresponding solutions or directions. We also discuss the challenges and possible research directions of the Metaverse. We believe this comprehensive review can provide an overview of the Metaverse and research directions for some multidisciplinary studies.
Novel 3D Scene Understanding Applications From Recurrence in a Single Image
Venue: arXiv cs.CV 2022
Year: 2022

We demonstrate the utility of recurring pattern discovery from a single image for spatial understanding of a 3D scene in terms of (1) vanishing point detection, (2) hypothesizing 3D translation symmetry and (3) counting the number of RP instances in the image. Furthermore, we illustrate the feasibility of leveraging RP discovery output to form a more precise, quantitative text description of the scene. Our quantitative evaluations on a new 1K+ Recurring Pattern (RP) benchmark with diverse variations show that visual perception of recurrence from one single view leads to scene understanding outcomes that are as good as or better than existing supervised methods and/or unsupervised methods that use millions of images.
Will Emerging Millimeter-Wave Cellular Networks Cause Harmful Interference to Weather Satellites?
Venue: arXiv cs.NI 2022
Year: 2022

We study whether realistic 5G mm-wave cellular networks would cause harmful out-of-band interference to weather satellites sensing in the 23.8 GHz band. We estimate uplink and downlink interference from a single interferer and a network of interferers in New York City, using real 3D building data and realistic antenna patterns. We perform detailed ray-tracing propagation simulations, for locations of the MetOp-B weather satellite and its scanning orientations and ground interferer antenna orientations for representative urban cell sites. In addition to the ITU-R threshold of -136 dBm/200 MHz, we propose an alternative set of harmful interference thresholds directly related to the sensitivity of the satellite sensor. Our results show that the 3GPP power leakage limits are sufficient to ensure that interference from a single 5G device is not harmful if considering the ITU-R threshold, but not if the weather prediction software can tolerate only very low interference levels. Importantly, aggregate interference resulting in practice from a 5G network with realistic network densities is often harmful, even considering the least conservative ITU-R threshold. Overall, our comprehensive coexistence study thus strongly suggests that additional engineering and/or regulatory solutions will be necessary to protect weather satellite passive sensing from mm-wave cellular network interference.
Word Clouds in the Wild
Venue: arXiv cs.HC 2022
Year: 2022

Word clouds are frequently used to analyze and communicate text data in many domains. In order to help guide research on improving the legibility of word clouds, we have conducted a survey of their usage in Digital Humanities academia and journalism. Using a modified grounded theory approach, we sought to identify the most common purposes for which word clouds were employed and the most common visual encodings they contained. Our findings indicate that font size, color, and word placement dominate as the primary data-encoding channels, as we hypothesized. Perhaps more surprisingly, we found that asking viewers to perform analytical tasks with word clouds was relatively common, especially in DH sources. This suggests that research into the interactions of these visual encoding channels (particularly in regards to legibility) is warranted.
Eliciting Compatible Demonstrations for Multi-Human Imitation Learning
Venue: arXiv cs.RO 2022
Year: 2022

Imitation learning from human-provided demonstrations is a strong approach for learning policies for robot manipulation. While the ideal dataset for imitation learning is homogenous and low-variance -- reflecting a single, optimal method for performing a task -- natural human behavior has a great deal of heterogeneity, with several optimal ways to demonstrate a task. This multimodality is inconsequential to human users, with task variations manifesting as subconscious choices; for example, reaching down, then across to grasp an object, versus reaching across, then down. Yet, this mismatch presents a problem for interactive imitation learning, where sequences of users improve on a policy by iteratively collecting new, possibly conflicting demonstrations. To combat this problem of demonstrator incompatibility, this work designs an approach for 1) measuring the compatibility of a new demonstration given a base policy, and 2) actively eliciting more compatible demonstrations from new users. Across two simulation tasks requiring long-horizon, dexterous manipulation and a real-world "food plating" task with a Franka Emika Panda arm, we show that we can both identify incompatible demonstrations via post-hoc filtering, and apply our compatibility measure to actively elicit compatible demonstrations from new users, leading to improved task success rates across simulated and real environments.
Reference Based Color Transfer for Medical Volume Rendering
Venue: arXiv cs.CV 2022
Year: 2022

The benefits of medical imaging are enormous. Medical images provide considerable amounts of anatomical information and this facilitates medical practitioners in performing effective disease diagnosis and deciding upon the best course of medical treatment. A transition from traditional monochromatic medical images like CT scans, X-Rays or MRI images to a colored 3D representation of the anatomical structure further enhances the capabilities of medical professionals in extracting valuable medical information. The proposed framework in our research starts with performing color transfer by finding deep semantic correspondence between two medical images: a colored reference image, and a monochromatic CT scan or an MRI image. We extend this idea of reference-based colorization technique to perform colored volume rendering from a stack of grayscale medical images. Furthermore, we also propose to use an effective reference image recommendation system to aid in the selection of good reference images. With our approach, we successfully perform colored medical volume visualization and essentially eliminate the painstaking process of user interaction with a transfer function to obtain color and opacity parameters for volume rendering.
Knowledge Distillation approach towards Melanoma Detection
Venue: arXiv cs.CV 2022
Year: 2022

Melanoma is regarded as the most threatening among all skin cancers. There is a pressing need to build systems which can aid in the early detection of melanoma and enable timely treatment to patients. Recent methods are geared towards machine learning based systems where the task is posed as image recognition, tag dermoscopic images of skin lesions as melanoma or non-melanoma. Even though these methods show promising results in terms of accuracy, they are computationally quite expensive to train, that questions the ability of these models to be deployable in a clinical setting or memory constraint devices. To address this issue, we focus on building simple and performant models having few layers, less than ten compared to hundreds. As well as with fewer learnable parameters, 0.26 million (M) compared to 42.5M using knowledge distillation with the goal to detect melanoma from dermoscopic images. First, we train a teacher model using a ResNet-50 to detect melanoma. Using the teacher model, we train the student model known as Distilled Student Network (DSNet) which has around 0.26M parameters using knowledge distillation achieving an accuracy of 91.7%. We compare against ImageNet pre-trained models such MobileNet, VGG-16, Inception-V3, EfficientNet-B0, ResNet-50 and ResNet-101. We find that our approach works well in terms of inference runtime compared to other pre-trained models, 2.57 seconds compared to 14.55 seconds. We find that DSNet (0.26M parameters), which is 15 times smaller, consistently performs better than EfficientNet-B0 (4M parameters) in both melanoma and non-melanoma detection across Precision, Recall and F1 scores
Hidden Complexities in the Computational Modeling of Proportionality for Robotic Norm Violation Response
Venue: arXiv cs.RO 2022
Year: 2022

Language-capable robots hold unique persuasive power over humans, and thus can help regulate people's behavior and preserve a better moral ecosystem, by rejecting unethical commands and calling out norm violations. However, miscalibrated norm violation responses (when the harshness of a response does not match the actual norm violation severity) may not only decrease the effectiveness of human-robot communication, but may also damage the rapport between humans and robots. Therefore, when robots respond to norm violations, it is crucial that they consider both the moral value of their response (by considering how much positive moral influence their response could exert) and the social value (by considering how much face threat might be imposed by their utterance). In this paper, we present a simple (naive) mathematical model of proportionality which could explain how moral and social considerations should be balanced in multi-agent norm violation response generation. But even more importantly, we use this model to start a discussion about the hidden complexity of modeling proportionality, and use this discussion to identify key research directions that must be explored in order to develop socially and morally competent language-capable robots.
Temporal Word Meaning Disambiguation using TimeLMs
Venue: arXiv cs.CL 2022
Year: 2022

Meaning of words constantly changes given the events in modern civilization. Large Language Models use word embeddings, which are often static and thus cannot cope with this semantic change. Thus,it is important to resolve ambiguity in word meanings. This paper is an effort in this direction, where we explore methods for word sense disambiguation for the EvoNLP shared task. We conduct rigorous ablations for two solutions to this problem. We see that an approach using time-aware language models helps this task. Furthermore, we explore possible future directions to this problem.
PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scale
Venue: arXiv cs.RO 2022
Year: 2022

The predictive information, the mutual information between the past and future, has been shown to be a useful representation learning auxiliary loss for training reinforcement learning agents, as the ability to model what will happen next is critical to success on many control tasks. While existing studies are largely restricted to training specialist agents on single-task settings in simulation, in this work, we study modeling the predictive information for robotic agents and its importance for general-purpose agents that are trained to master a large repertoire of diverse skills from large amounts of data. Specifically, we introduce Predictive Information QT-Opt (PI-QT-Opt), a QT-Opt agent augmented with an auxiliary loss that learns representations of the predictive information to solve up to 297 vision-based robot manipulation tasks in simulation and the real world with a single set of parameters. We demonstrate that modeling the predictive information significantly improves success rates on the training tasks and leads to better zero-shot transfer to unseen novel tasks. Finally, we evaluate PI-QT-Opt on real robots, achieving substantial and consistent improvement over QT-Opt in multiple experimental settings of varying environments, skills, and multi-task configurations.