OpenRaiser/Intern-Atlas huggingface dataset:
NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to Robotics
Venue: arXiv cs.RO 2023
Year: 2023

This work was presented at the IEEE International Conference on Robotics and Automation 2023 Workshop on Unconventional Spatial Representations. Neural radiance fields (NeRFs) are a class of implicit scene representations that model 3D environments from color images. NeRFs are expressive, and can model the complex and multi-scale geometry of real world environments, which potentially makes them a powerful tool for robotics applications. Modern NeRF training libraries can generate a photo-realistic NeRF from a static data set in just a few seconds, but are designed for offline use and require a slow pose optimization pre-computation step. In this work we propose NerfBridge, an open-source bridge between the Robot Operating System (ROS) and the popular Nerfstudio library for real-time, online training of NeRFs from a stream of images. NerfBridge enables rapid development of research on applications of NeRFs in robotics by providing an extensible interface to the efficient training pipelines and model libraries provided by Nerfstudio. As an example use case we outline a hardware setup that can be used NerfBridge to train a NeRF from images captured by a camera mounted to a quadrotor in both indoor and outdoor environments. For accompanying video https://youtu.be/EH0SLn-RcDg and code https://github.com/javieryu/nerf_bridge.
A Comparative Study of GAN-Generated Handwriting Images and MNIST Images using t-SNE Visualization
Venue: arXiv cs.CV 2023
Year: 2023

The quality of GAN-generated images on the MNIST dataset was explored in this paper by comparing them to the original images using t-distributed stochastic neighbor embedding (t- SNE) visualization. A GAN was trained with the dataset to generate images and the result of generating all synthetic images, the corresponding labels were saved. The dimensionality of the generated images and the original MNIST dataset was reduced using t-SNE and the resulting embeddings were plotted. The rate of the GAN-generated images was examined by comparing the t-SNE plots of the generated images and the original MNIST images. It was found that the GAN- generated images were similar to the original images but had some differences in the distribution of the features. It is believed that this study provides a useful evaluation method for assessing the quality of GAN-generated images and can help to improve their generation in the future.
Poster: No safety in numbers: traffic analysis of sealed-sender groups in Signal
Venue: arXiv cs.CR 2023
Year: 2023

Secure messaging applications often offer privacy to users by protecting their messages from would be observers through end-to-end encryption techniques. However, the metadata of who communicates with whom cannot be concealed by encryption alone. Signal's Sealed Sender mechanism attempts to enhance its protection of this data by obfuscating the sender of any message sent with the protocol. However, it was shown by Martiny et al. that due to the message delivery protocols in Signal, the record of who receives messages can be enough to recover this metadata. In this work we extend the attack presented from deanonymizing communicating pairs to deanonymizing entire group conversations.
Semi-Supervised Object Detection for Sorghum Panicles in UAV Imagery
Venue: arXiv cs.CV 2023
Year: 2023

The sorghum panicle is an important trait related to grain yield and plant development. Detecting and counting sorghum panicles can provide significant information for plant phenotyping. Current deep-learning-based object detection methods for panicles require a large amount of training data. The data labeling is time-consuming and not feasible for real application. In this paper, we present an approach to reduce the amount of training data for sorghum panicle detection via semi-supervised learning. Results show we can achieve similar performance as supervised methods for sorghum panicle detection by only using 10\% of original training data.
Single-Photon Counting Receivers for Optical Wireless Communications in Future 6G Networks
Venue: arXiv cs.IT 2023
Year: 2023

Optical wireless communication (OWC) offers several complementary advantages to radio-frequency wireless networks such as its massive available spectrum; hence, it is widely anticipated that OWC will assume a pivotal role in the forthcoming sixth generation wireless communication networks. Although significant progress has been achieved in OWC over the past decades, the outage induced by occasionally low received optical power continues to pose a key limiting factor for its deployment. In this work, we discuss the potential role of single-photon counting (SPC) receivers as a promising solution to overcome this limitation. We present an overview of the applications of SPC-based OWC systems in 6G networks, introduce their major performance-limiting factors, propose a performance enhancement framework to tackle these issues, and identify critical areas of open problems for future research.
Generative Model-based Simulation of Driver Behavior when Using Control Input Interface for Teleoperated Driving in Unstructured Canyon Terrains
Venue: arXiv cs.RO 2023
Year: 2023

Unmanned ground vehicles (UGVs) in unstructured environments mostly operate through teleoperation. To enable stable teleoperated driving in unstructured environments, some research has suggested driver assistance and evaluation methods that involve user studies, which can be costly and require lots of time and effort. A simulation model-based approach has been proposed to complement the user study; however, the models on teleoperated driving do not account for unstructured environments. Our proposed solution involves simulation models of teleoperated driving for drivers that utilize a deep generative model. Initially, we build a teleoperated driving simulator to imitate unstructured environments based on previous research and collect driving data from drivers. Then, we design and implement the simulation models based on a conditional variational autoencoder (CVAE). Our evaluation results demonstrate that the proposed teleoperated driving model can generate data by simulating the driver appropriately in unstructured canyon terrains.
Augmenting Learning with Augmented Reality: Exploring the Affordances of AR in Supporting Mastery of Complex Psychomotor Tasks
Venue: arXiv cs.HC 2023
Year: 2023

This research seeks to explore how Augmented Reality (AR) can support learning psychomotor tasks that involve complex manipulation and reasoning processes. The AR prototype was created using Unity and used on HoloLens 2 headsets. Here, we explore the potential of AR as a training or assistive tool for spatial tasks and the need for intelligent mechanisms to enable adaptive and personalized interactions between learners and AR. The paper discusses how integrating AR with Artificial Intelligence (AI) can adaptably scaffold the learning of complex tasks to accelerate the development of expertise in psychomotor domains.
Semantic Similarity Measure of Natural Language Text through Machine Learning and a Keyword-Aware Cross-Encoder-Ranking Summarizer -- A Case Study Using UCGIS GIS&T Body of Knowledge
Venue: arXiv cs.CL 2023
Year: 2023

Initiated by the University Consortium of Geographic Information Science (UCGIS), GIS&T Body of Knowledge (BoK) is a community-driven endeavor to define, develop, and document geospatial topics related to geographic information science and technologies (GIS&T). In recent years, GIS&T BoK has undergone rigorous development in terms of its topic re-organization and content updating, resulting in a new digital version of the project. While the BoK topics provide useful materials for researchers and students to learn about GIS, the semantic relationships among the topics, such as semantic similarity, should also be identified so that a better and automated topic navigation can be achieved. Currently, the related topics are either defined manually by editors or authors, which may result in an incomplete assessment of topic relationship. To address this challenge, our research evaluates the effectiveness of multiple natural language processing (NLP) techniques in extracting semantics from text, including both deep neural networks and traditional machine learning approaches. Besides, a novel text summarization - KACERS (Keyword-Aware Cross-Encoder-Ranking Summarizer) - is proposed to generate a semantic summary of scientific publications. By identifying the semantic linkages among key topics, this work provides guidance for future development and content organization of the GIS&T BoK project. It also offers a new perspective on the use of machine learning techniques for analyzing scientific publications, and demonstrate the potential of KACERS summarizer in semantic understanding of long text documents.
Analyzing the Stance of Facebook Posts on Abortion Considering State-level Health and Social Compositions
Venue: arXiv cs.SI 2023
Year: 2023

Abortion remains one of the most controversial topics, especially after overturning Roe v. Wade ruling in the United States. Previous literature showed that the illegality of abortion could have serious consequences, as women might seek unsafe pregnancy terminations leading to increased maternal mortality rates and negative effects on their reproductive health. Therefore, the stances of the abortion-related Facebook posts were analyzed at the state level in the United States from May 4 until June 30, 2022, right after the Supreme Court's decision was disclosed. In more detail, the pre-trained Transformer architecture-based model was fine-tuned on a manually labeled training set to obtain a stance detection model suitable for the collected dataset. Afterward, we employed appropriate statistical tests to examine the relationships between public opinion regarding abortion, abortion legality, political leaning, and factors measuring the overall population's health, health knowledge, and vulnerability per state. We found that states with a higher number of views against abortion also have higher infant and maternal mortality rates. Furthermore, the stance of social media posts per state is mostly matching with the current abortion laws in these states. While aligned with existing literature, these findings indicate how public opinion, laws, and women's and infants' health are related, and interventions are required to educate and protect women, especially in vulnerable populations.
Assessing the Impact of Context Inference Error and Partial Observability on RL Methods for Just-In-Time Adaptive Interventions
Venue: arXiv cs.LG 2023
Year: 2023

Just-in-Time Adaptive Interventions (JITAIs) are a class of personalized health interventions developed within the behavioral science community. JITAIs aim to provide the right type and amount of support by iteratively selecting a sequence of intervention options from a pre-defined set of components in response to each individual's time varying state. In this work, we explore the application of reinforcement learning methods to the problem of learning intervention option selection policies. We study the effect of context inference error and partial observability on the ability to learn effective policies. Our results show that the propagation of uncertainty from context inferences is critical to improving intervention efficacy as context uncertainty increases, while policy gradient algorithms can provide remarkable robustness to partially observed behavioral state information.
Impact of ROS 2 Node Composition in Robotic Systems
Venue: arXiv cs.RO 2023
Year: 2023

The Robot Operating System 2 (ROS 2) is the second generation of ROS representing a step forward in the robotic framework. Several new types of nodes and executor models are integral to control where, how, and when information is processed in the computational graph. This paper explores and benchmarks one of these new node types -- the Component node -- which allows nodes to be composed manually or dynamically into processes while retaining separation of concerns in a codebase for distributed development. Composition is shown to achieve a high degree of performance optimization, particularly valuable for resource-constrained systems and sensor processing pipelines, enabling distributed tasks that would not be otherwise possible in ROS 2. In this work, we briefly introduce the significance and design of node composition, then our contribution of benchmarking is provided to analyze its impact on robotic systems. Its compelling influence on performance is shown through several experiments on the latest Long Term Support (LTS) ROS 2 distribution, Humble Hawksbill.
Link prediction for ex ante influence maximization on temporal networks
Venue: arXiv cs.SI 2023
Year: 2023

Influence maximization (IM) is the task of finding the most important nodes in order to maximize the spread of influence or information on a network. This task is typically studied on static or temporal networks where the complete topology of the graph is known. In practice, however, the seed nodes must be selected before observing the future evolution of the network. In this work, we consider this realistic ex ante setting where $p$ time steps of the network have been observed before selecting the seed nodes. Then the influence is calculated after the network continues to evolve for a total of $T>p$ time steps. We address this problem by using statistical, non-negative matrix factorization and graph neural networks link prediction algorithms to predict the future evolution of the network and then apply existing influence maximization algorithms on the predicted networks. Additionally, the output of the link prediction methods can be used to construct novel IM algorithms. We apply the proposed methods to eight real-world and synthetic networks to compare their performance using the Susceptible-Infected (SI) diffusion model. We demonstrate that it is possible to construct quality seed sets in the ex ante setting as we achieve influence spread within 87\% of the optimal spread on seven of eight network. In many settings, choosing seed nodes based only historical edges provides results comparable to the results treating the future graph snapshots as known. The proposed heuristics based on the link prediction model are also some of the best-performing methods. These findings indicate that, for these eight networks under the SI model, the latent process which determines the most influential nodes may not have large temporal variation. Thus, knowing the future status of the network is not necessary to obtain good results for ex ante IM.
Singly Exponential Translation of Alternating Weak Büchi Automata to Unambiguous Büchi Automata
Venue: arXiv cs.FL 2023
Year: 2023

We introduce a method for translating an alternating weak Büchi automaton (AWA), which corresponds to a Linear Dynamic Logic (LDL) formula, to an unambiguous Büchi automaton (UBA). Our translations generalise constructions for Linear Temporal Logic (LTL), a less expressive specification language than LDL. In classical constructions, LTL formulas are first translated to alternating \emph{very weak} automata (AVAs) -- automata that have only singleton strongly connected components (SCCs); the AVAs are then handled by efficient disambiguation procedures. However, general AWAs can have larger SCCs, which complicates disambiguation. Currently, the only available disambiguation procedure has to go through an intermediate construction of nondeterministic Büchi automata (NBAs), which would incur an exponential blow-up of its own. We introduce a translation from \emph{general} AWAs to UBAs with a \emph{singly} exponential blow-up, which also immediately provides a singly exponential translation from LDL to UBAs. Interestingly, the complexity of our translation is \emph{smaller} than the best known disambiguation algorithm for NBAs (broadly $(0.53n)^n$ vs. $(0.76n)^n$), while the input of our construction can be exponentially more succinct.
DRackSim: Simulator for Rack-scale Memory Disaggregation
Venue: arXiv cs.DC 2023
Year: 2023

Memory disaggregation has emerged as an alternative to traditional server architecture in data centers. This paper introduces DRackSim, a simulation infrastructure to model rack-scale hardware disaggregated memory. DRackSim models multiple compute nodes, memory pools, and a rack-scale interconnect similar to GenZ. An application-level simulation approach simulates an x86 out-of-order multi-core processor with a multi-level cache hierarchy at compute nodes. A queue-based simulation is used to model a remote memory controller and rack-level interconnect, which allows both cache-based and page-based access to remote memory. DRackSim models a central memory manager to manage address space at the memory pools. We integrate community-accepted DRAMSim2 to perform memory simulation at local and remote memory using multiple DRAMSim2 instances. An incremental approach is followed to validate the core and cache subsystem of DRackSim with that of Gem5. We measure the performance of various HPC workloads and show the performance impact for different nodes/pools configuration.
The Noncommutative Edmonds' Problem Re-visited
Venue: arXiv cs.CC 2023
Year: 2023

Let $T$ be a matrix whose entries are linear forms over the noncommutative variables $x_1, x_2, \ldots, x_n$. The noncommutative Edmonds' problem (NSINGULAR) aims to determine whether $T$ is invertible in the free skew field generated by $x_1,x_2,\ldots,x_n$. Currently, there are three different deterministic polynomial-time algorithms to solve this problem: using operator scaling [Garg, Gurvits, Oliveira, and Wigserdon (2016)], algebraic methods [Ivanyos, Qiao, and Subrahmanyam (2018)], and convex optimization [Hamada and Hirai (2021)]. In this paper, we present a simpler algorithm for the NSINGULAR problem. While our algorithmic template is similar to the one in Ivanyos et. al.(2018), it significantly differs in its implementation of the rank increment step. Instead of computing the limit of a second Wong sequence, we reduce the problem to the polynomial identity testing (PIT) of noncommutative algebraic branching programs (ABPs). This enables us to bound the bit-complexity of the algorithm over $\mathbb{Q}$ without requiring special care. Moreover, the rank increment step can be implemented in quasipolynomial-time even without an explicit description of the coefficient matrices in $T$. This is possible by exploiting the connection with the black-box PIT of noncommutative ABPs [Forbes and Shpilka (2013)].
When Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario
Venue: arXiv cs.CL 2023
Year: 2023

Large pre-trained language models (PLMs) have garnered significant attention for their versatility and potential for solving a wide spectrum of natural language processing (NLP) tasks. However, the cost of running these PLMs may be prohibitive. Furthermore, PLMs may not be open-sourced due to commercial considerations and potential risks of misuse, such as GPT-3. The parameters and gradients of PLMs are unavailable in this scenario. To solve the issue, black-box tuning has been proposed, which utilizes derivative-free optimization (DFO), instead of gradient descent, for training task-specific continuous prompts. However, these gradient-free methods still exhibit a significant gap compared to gradient-based methods. In this paper, we introduce gradient descent into black-box tuning scenario through knowledge distillation. Furthermore, we propose a novel method GDFO, which integrates gradient descent and derivative-free optimization to optimize task-specific continuous prompts in a harmonized manner. Experimental results show that GDFO can achieve significant performance gains over previous state-of-the-art methods.
TextSLAM: Visual SLAM with Semantic Planar Text Features
Venue: arXiv cs.CV 2023
Year: 2023

We propose a novel visual SLAM method that integrates text objects tightly by treating them as semantic features via fully exploring their geometric and semantic prior. The text object is modeled as a texture-rich planar patch whose semantic meaning is extracted and updated on the fly for better data association. With the full exploration of locally planar characteristics and semantic meaning of text objects, the SLAM system becomes more accurate and robust even under challenging conditions such as image blurring, large viewpoint changes, and significant illumination variations (day and night). We tested our method in various scenes with the ground truth data. The results show that integrating texture features leads to a more superior SLAM system that can match images across day and night. The reconstructed semantic 3D text map could be useful for navigation and scene understanding in robotic and mixed reality applications. Our project page: https://github.com/SJTU-ViSYS/TextSLAM .
SHoP: A Deep Learning Framework for Solving High-order Partial Differential Equations
Venue: arXiv cs.LG 2023
Year: 2023

Solving partial differential equations (PDEs) has been a fundamental problem in computational science and of wide applications for both scientific and engineering research. Due to its universal approximation property, neural network is widely used to approximate the solutions of PDEs. However, existing works are incapable of solving high-order PDEs due to insufficient calculation accuracy of higher-order derivatives, and the final network is a black box without explicit explanation. To address these issues, we propose a deep learning framework to solve high-order PDEs, named SHoP. Specifically, we derive the high-order derivative rule for neural network, to get the derivatives quickly and accurately; moreover, we expand the network into a Taylor series, providing an explicit solution for the PDEs. We conduct experimental validations four high-order PDEs with different dimensions, showing that we can solve high-order PDEs efficiently and accurately.
Two-Stream Regression Network for Dental Implant Position Prediction
Venue: arXiv cs.CV 2023
Year: 2023

In implant prosthesis treatment, the design of the surgical guide heavily relies on the manual location of the implant position, which is subjective and prone to doctor's experiences. When deep learning based methods has started to be applied to address this problem, the space between teeth are various and some of them might present similar texture characteristic with the actual implant region. Both problems make a big challenge for the implant position prediction. In this paper, we develop a two-stream implant position regression framework (TSIPR), which consists of an implant region detector (IRD) and a multi-scale patch embedding regression network (MSPENet), to address this issue. For the training of IRD, we extend the original annotation to provide additional supervisory information, which contains much more rich characteristic and do not introduce extra labeling costs. A multi-scale patch embedding module is designed for the MSPENet to adaptively extract features from the images with various tooth spacing. The global-local feature interaction block is designed to build the encoder of MSPENet, which combines the transformer and convolution for enriched feature representation. During inference, the RoI mask extracted from the IRD is used to refine the prediction results of the MSPENet. Extensive experiments on a dental implant dataset through five-fold cross-validation demonstrated that the proposed TSIPR achieves superior performance than existing methods.