OpenRaiser/Intern-Atlas huggingface dataset:
The Impact of Different Backbone Architecture on Autonomous Vehicle Dataset
Venue: arXiv cs.CV 2023
Year: 2023

Object detection is a crucial component of autonomous driving, and many detection applications have been developed to address this task. These applications often rely on backbone architectures, which extract representation features from inputs to perform the object detection task. The quality of the features extracted by the backbone architecture can have a significant impact on the overall detection performance. Many researchers have focused on developing new and improved backbone architectures to enhance the efficiency and accuracy of object detection applications. While these backbone architectures have shown state-of-the-art performance on generic object detection datasets like MS-COCO and PASCAL-VOC, evaluating their performance under an autonomous driving environment has not been previously explored. To address this, our study evaluates three well-known autonomous vehicle datasets, namely KITTI, NuScenes, and BDD, to compare the performance of different backbone architectures on object detection tasks.
Sparse Autoencoders Find Highly Interpretable Features in Language Models
Venue: arXiv cs.LG 2023
Year: 2023

One of the roadblocks to a better understanding of neural networks' internals is \textit{polysemanticity}, where neurons appear to activate in multiple, semantically distinct contexts. Polysemanticity prevents us from identifying concise, human-understandable explanations for what neural networks are doing internally. One hypothesised cause of polysemanticity is \textit{superposition}, where neural networks represent more features than they have neurons by assigning features to an overcomplete set of directions in activation space, rather than to individual neurons. Here, we attempt to identify those directions, using sparse autoencoders to reconstruct the internal activations of a language model. These autoencoders learn sets of sparsely activating features that are more interpretable and monosemantic than directions identified by alternative approaches, where interpretability is measured by automated methods. Moreover, we show that with our learned set of features, we can pinpoint the features that are causally responsible for counterfactual behaviour on the indirect object identification task \citep{wang2022interpretability} to a finer degree than previous decompositions. This work indicates that it is possible to resolve superposition in language models using a scalable, unsupervised method. Our method may serve as a foundation for future mechanistic interpretability work, which we hope will enable greater model transparency and steerability.
Maneuver Decision-Making Through Proximal Policy Optimization And Monte Carlo Tree Search
Venue: arXiv cs.AI 2023
Year: 2023

Maneuver decision-making can be regarded as a Markov decision process and can be address by reinforcement learning. However, original reinforcement learning algorithms can hardly solve the maneuvering decision-making problem. One reason is that agents use random actions in the early stages of training, which makes it difficult to get rewards and learn how to make effective decisions. To address this issue, a method based on proximal policy optimization and Monte Carlo tree search is proposed. The method uses proximal policy optimization to train the agent, and regards the results of air combat as targets to train the value network. Then, based on the value network and the visit count of each node, Monte Carlo tree search is used to find the actions with more expected returns than random actions, which can improve the training performance. The ablation studies and simulation experiments indicate that agents trained by the proposed method can make different decisions according to different states, which demonstrates that the method can solve the maneuvering decision problem that the original reinforcement learning algorithm cannot solve.
Superpositioner -- A non-logical computation model
Venue: arXiv cs.CC 2023
Year: 2023

We have been striving to exceed computational complexity, and in the process, we have come to realize the dilemma of classical computing, and in turn we realize that superpositioner may be a way to solve. A superpositioner is a model formed by several Boolean functions that their variables and function values are feeding back to each other. The component of the superpositioner is the reentry function, which can be fully described by classical logic and can be calculated by classical computation, but the superpositioner as a whole is a non-logical entity, and it is impossible for classical computation to fully compute it. In this article, we present the concept of a superpositioner and discuss its basic properties. We find that the superpositioner + dispositioner will form a new type of computation model whose capabilities can surpass Turing computation. We envision that this new model will help implement these functions in the intelligent agent: a whole new way of programming and a whole new way of learning, endogenous feelings, analogies and associations, forming understanding, dynamic action, participating in the formation of subjectivity, and more. We will also discuss how to implement superpositioner in the most preliminary way.
An inspection technology of inner surface of the fine hole based on machine vision
Venue: arXiv cs.CV 2023
Year: 2023

Fine holes are an important structural component of industrial components, and their inner surface quality is closely related to their function.In order to detect the quality of the inner surface of the fine hole,a special optical measurement system was investigated in this paper. A sight pipe is employed to guide the external illumination light into the fine hole and output the relevant images simultaneously. A flexible light array is introduced to suit the narrow space, and the effective field of view is analyzed. Besides, the arc surface projection error and manufacturing assembly error of the device are analyzed, then compensated or ignored if small enough. In the test of prefabricated circular defects with the diameter ϕ0.1mm, ϕ0.2mm, 0.4mm distance distribution and the fissure defects with the width 0.3mm, the maximum measurement error standard deviation are all about 10μm. The minimum diameter of the measured fine hole is 4mm and the depth can reach 47mm.
Deterministic Real-Time Tree-Walking-Storage Automata
Venue: arXiv cs.FL 2023
Year: 2023

We study deterministic tree-walking-storage automata, which are finite-state devices equipped with a tree-like storage. These automata are generalized stack automata, where the linear stack storage is replaced by a non-linear tree-like stack. Therefore, tree-walking-storage automata have the ability to explore the interior of the tree storage without altering the contents, with the possible moves of the tree pointer corresponding to those of tree-walking automata. In addition, a tree-walking-storage automaton can append (push) non-existent descendants to a tree node and remove (pop) leaves from the tree. Here we are particularly considering the capacities of deterministic tree-walking-storage automata working in real time. It is shown that even the non-erasing variant can accept rather complicated unary languages as, for example, the language of words whose lengths are powers of two, or the language of words whose lengths are Fibonacci numbers. Comparing the computational capacities with automata from the classical automata hierarchy, we derive that the families of languages accepted by real-time deterministic (non-erasing) tree-walking-storage automata is located between the regular and the deterministic context-sensitive languages. There is a context-free language that is not accepted by any real-time deterministic tree-walking-storage automaton. On the other hand, these devices accept a unary language in non-erasing mode that cannot be accepted by any classical stack automaton, even in erasing mode and arbitrary time. Basic closure properties of the induced families of languages are shown. In particular, we consider Boolean operations (complementation, union, intersection) and AFL operations (union, intersection with regular languages, homomorphism, inverse homomorphism, concatenation, iteration). It turns out that the two families in question have the same properties and, in particular, share all but one of these closure properties with the important family of deterministic context-free languages.
Personalized Food Image Classification: Benchmark Datasets and New Baseline
Venue: arXiv cs.CV 2023
Year: 2023

Food image classification is a fundamental step of image-based dietary assessment, enabling automated nutrient analysis from food images. Many current methods employ deep neural networks to train on generic food image datasets that do not reflect the dynamism of real-life food consumption patterns, in which food images appear sequentially over time, reflecting the progression of what an individual consumes. Personalized food classification aims to address this problem by training a deep neural network using food images that reflect the consumption pattern of each individual. However, this problem is under-explored and there is a lack of benchmark datasets with individualized food consumption patterns due to the difficulty in data collection. In this work, we first introduce two benchmark personalized datasets including the Food101-Personal, which is created based on surveys of daily dietary patterns from participants in the real world, and the VFNPersonal, which is developed based on a dietary study. In addition, we propose a new framework for personalized food image classification by leveraging self-supervised learning and temporal image feature information. Our method is evaluated on both benchmark datasets and shows improved performance compared to existing works. The dataset has been made available at: https://skynet.ecn.purdue.edu/~pan161/dataset_personal.html
Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representations
Venue: arXiv cs.CV 2023
Year: 2023

We introduce MHVAE, a deep hierarchical variational auto-encoder (VAE) that synthesizes missing images from various modalities. Extending multi-modal VAEs with a hierarchical latent structure, we introduce a probabilistic formulation for fusing multi-modal images in a common latent representation while having the flexibility to handle incomplete image sets as input. Moreover, adversarial learning is employed to generate sharper images. Extensive experiments are performed on the challenging problem of joint intra-operative ultrasound (iUS) and Magnetic Resonance (MR) synthesis. Our model outperformed multi-modal VAEs, conditional GANs, and the current state-of-the-art unified method (ResViT) for synthesizing missing images, demonstrating the advantage of using a hierarchical latent representation and a principled probabilistic fusion operation. Our code is publicly available \url{https://github.com/ReubenDo/MHVAE}.
Biased Attention: Do Vision Transformers Amplify Gender Bias More than Convolutional Neural Networks?
Venue: arXiv cs.CV 2023
Year: 2023

Deep neural networks used in computer vision have been shown to exhibit many social biases such as gender bias. Vision Transformers (ViTs) have become increasingly popular in computer vision applications, outperforming Convolutional Neural Networks (CNNs) in many tasks such as image classification. However, given that research on mitigating bias in computer vision has primarily focused on CNNs, it is important to evaluate the effect of a different network architecture on the potential for bias amplification. In this paper we therefore introduce a novel metric to measure bias in architectures, Accuracy Difference. We examine bias amplification when models belonging to these two architectures are used as a part of large multimodal models, evaluating the different image encoders of Contrastive Language Image Pretraining which is an important model used in many generative models such as DALL-E and Stable Diffusion. Our experiments demonstrate that architecture can play a role in amplifying social biases due to the different techniques employed by the models for feature extraction and embedding as well as their different learning properties. This research found that ViTs amplified gender bias to a greater extent than CNNs
The Fractal Hand-II: Reviving a Classic Mechanism for Contemporary Grasping Challenges
Venue: arXiv cs.RO 2023
Year: 2023

This paper, and its companion, propose a new fractal robotic gripper, drawing inspiration from the century-old Fractal Vise. The unusual synergistic properties allow it to passively conform to diverse objects using only one actuator. Designed to be easily integrated with prevailing parallel jaw grippers, it alleviates the complexities tied to perception and grasp planning, especially when dealing with unpredictable object poses and geometries. We build on the foundational principles of the Fractal Vise to a broader class of gripping mechanisms, and also address the limitations that had led to its obscurity. Two Fractal Fingers, coupled by a closing actuator, can form an adaptive and synergistic Fractal Hand. We articulate a design methodology for low cost, easy to fabricate, large workspace, and compliant Fractal Fingers. The companion paper delves into the kinematics and grasping properties of a specific class of Fractal Fingers and Hands.
Geometric Projectors: Geometric Constraints based Optimization for Robot Behaviors
Venue: arXiv cs.RO 2023
Year: 2023

Generating motion for robots that interact with objects of various shapes is a complex challenge, further complicated when the robot's own geometry and multiple desired behaviors are considered. To address this issue, we introduce a new framework based on Geometric Projectors (GeoPro) for constrained optimization. This novel framework allows for the generation of task-agnostic behaviors that are compliant with geometric constraints. GeoPro streamlines the design of behaviors in both task and configuration spaces, offering diverse functionalities such as collision avoidance and goal-reaching, while maintaining high computational efficiency. We validate the efficacy of our work through simulations and Franka Emika robotic experiments, comparing its performance against state-of-the-art methodologies. This comprehensive evaluation highlights GeoPro's versatility in accommodating robots with varying dynamics and precise geometric shapes. For additional materials, please visit: https://www.xueminchi.com/publications/geopro
URA*: Uncertainty-aware Path Planning using Image-based Aerial-to-Ground Traversability Estimation for Off-road Environments
Venue: arXiv cs.RO 2023
Year: 2023

A major challenge with off-road autonomous navigation is the lack of maps or road markings that can be used to plan a path for autonomous robots. Classical path planning methods mostly assume a perfectly known environment without accounting for the inherent perception and sensing uncertainty from detecting terrain and obstacles in off-road environments. Recent work in computer vision and deep neural networks has advanced the capability of terrain traversability segmentation from raw images; however, the feasibility of using these noisy segmentation maps for navigation and path planning has not been adequately explored. To address this problem, this research proposes an uncertainty-aware path planning method, URA* using aerial images for autonomous navigation in off-road environments. An ensemble convolutional neural network (CNN) model is first used to perform pixel-level traversability estimation from aerial images of the region of interest. The traversability predictions are represented as a grid of traversal probability values. An uncertainty-aware planner is then applied to compute the best path from a start point to a goal point given these noisy traversal probability estimates. The proposed planner also incorporates replanning techniques to allow rapid replanning during online robot operation. The proposed method is evaluated on the Massachusetts Road Dataset, the DeepGlobe dataset, as well as a dataset of aerial images from off-road proving grounds at Mississippi State University. Results show that the proposed image segmentation and planning methods outperform conventional planning algorithms in terms of the quality and feasibility of the initial path, as well as the quality of replanned paths.
GPT as a Baseline for Recommendation Explanation Texts
Venue: arXiv cs.AI 2023
Year: 2023

In this work, we establish a baseline potential for how modern model-generated text explanations of movie recommendations may help users, and explore what different components of these text explanations that users like or dislike, especially in contrast to existing human movie reviews. We found that participants gave no significantly different rankings between movies, nor did they give significantly different individual quality scores to reviews of movies that they had never seen before. However, participants did mark reviews as significantly better when they were movies they had seen before. We also explore specific aspects of movie review texts that participants marked as important for each quality. Overall, we establish that modern LLMs are a promising source of recommendation explanations, and we intend on further exploring personalizable text explanations in the future.
A Categorical Framework for Program Semantics and Semantic Abstraction
Venue: arXiv cs.PL 2023
Year: 2023

Categorical semantics of type theories are often characterized as structure-preserving functors. This is because in category theory both the syntax and the domain of interpretation are uniformly treated as structured categories, so that we can express interpretations as structure-preserving functors between them. This mathematical characterization of semantics makes it convenient to manipulate and to reason about relationships between interpretations. Motivated by this success of functorial semantics, we address the question of finding a functorial analogue in abstract interpretation, a general framework for comparing semantics, so that we can bring similar benefits of functorial semantics to semantic abstractions used in abstract interpretation. Major differences concern the notion of interpretation that is being considered. Indeed, conventional semantics are value-based whereas abstract interpretation typically deals with more complex properties. In this paper, we propose a functorial approach to abstract interpretation and study associated fundamental concepts therein. In our approach, interpretations are expressed as oplax functors in the category of posets, and abstraction relations between interpretations are expressed as lax natural transformations representing concretizations. We present examples of these formal concepts from monadic semantics of programming languages and discuss soundness.
Measuring COVID-19 Related Media Consumption on Twitter
Venue: arXiv cs.SI 2023
Year: 2023

The COVID-19 pandemic has been affecting the world dramatically ever since 2020. The minimum availability of physical interactions during the lockdown has caused more and more people to turn to online activities on social media platforms. These platforms have provided essential updates regarding the pandemic, serving as bridges for communications. Research on studying these communications on different platforms emerges during the meantime. Prior studies focus on areas such as topic modeling, sentiment analysis and prediction tasks such as predicting COVID-19 positive cases, misinformation spread, etc. However, online communications with media outlets remain unexplored on an international scale. We have little knowledge about the patterns of the media consumption geographically and their association with offline political preference. We believe addressing these questions could help governments and researchers better understand human behaviors during the pandemic. In this thesis, we specifically investigate the online consumption of media outlets on Twitter through a set of quantitative analyses. We make use of several public media outlet datasets to extract media consumption from tweets collected based on COVID-19 keyword matching. We make use of a metric "interaction" to quantify media consumption through weighted Twitter activities. We further construct a matrix based on it which could be directly used to measure user-media consumption in different granularities. We then conduct analyses on the United States level and global level. To the best of our knowledge, this thesis presents the first-of-its-kind study on media consumption on COVID-19 across countries, it sheds light on understanding how people consume media outlets during the pandemic and provides potential insights for peer researchers.
ChatGPT-4 with Code Interpreter can be used to solve introductory college-level vector calculus and electromagnetism problems
Venue: arXiv cs.AI 2023
Year: 2023

We evaluated ChatGPT 3.5, 4, and 4 with Code Interpreter on a set of college-level engineering-math and electromagnetism problems, such as those often given to sophomore electrical engineering majors. We selected a set of 13 problems, and had ChatGPT solve them multiple times, using a fresh instance (chat) each time. We found that ChatGPT-4 with Code Interpreter was able to satisfactorily solve most problems we tested most of the time -- a major improvement over the performance of ChatGPT-4 (or 3.5) without Code Interpreter. The performance of ChatGPT was observed to be somewhat stochastic, and we found that solving the same problem N times in new ChatGPT instances and taking the most-common answer was an effective strategy. Based on our findings and observations, we provide some recommendations for instructors and students of classes at this level.
Robust Online Covariance and Sparse Precision Estimation Under Arbitrary Data Corruption
Venue: arXiv cs.LG 2023
Year: 2023

Gaussian graphical models are widely used to represent correlations among entities but remain vulnerable to data corruption. In this work, we introduce a modified trimmed-inner-product algorithm to robustly estimate the covariance in an online scenario even in the presence of arbitrary and adversarial data attacks. At each time step, data points, drawn nominally independently and identically from a multivariate Gaussian distribution, arrive. However, a certain fraction of these points may have been arbitrarily corrupted. We propose an online algorithm to estimate the sparse inverse covariance (i.e., precision) matrix despite this corruption. We provide the error-bound and convergence properties of the estimates to the true precision matrix under our algorithms.