OpenRaiser/Intern-Atlas huggingface dataset:
Spread Your Wings: A Radial Strip Transformer for Image Deblurring
Venue: arXiv cs.CV 2024
Year: 2024

Exploring motion information is important for the motion deblurring task. Recent the window-based transformer approaches have achieved decent performance in image deblurring. Note that the motion causing blurry results is usually composed of translation and rotation movements and the window-shift operation in the Cartesian coordinate system by the window-based transformer approaches only directly explores translation motion in orthogonal directions. Thus, these methods have the limitation of modeling the rotation part. To alleviate this problem, we introduce the polar coordinate-based transformer, which has the angles and distance to explore rotation motion and translation information together. In this paper, we propose a Radial Strip Transformer (RST), which is a transformer-based architecture that restores the blur images in a polar coordinate system instead of a Cartesian one. RST contains a dynamic radial embedding module (DRE) to extract the shallow feature by a radial deformable convolution. We design a polar mask layer to generate the offsets for the deformable convolution, which can reshape the convolution kernel along the radius to better capture the rotation motion information. Furthermore, we proposed a radial strip attention solver (RSAS) as deep feature extraction, where the relationship of windows is organized by azimuth and radius. This attention module contains radial strip windows to reweight image features in the polar coordinate, which preserves more useful information in rotation and translation motion together for better recovering the sharp images. Experimental results on six synthesis and real-world datasets prove that our method performs favorably against other SOTA methods for the image deblurring task.
Efficient Multi-branch Segmentation Network for Situation Awareness in Autonomous Navigation
Venue: arXiv cs.CV 2024
Year: 2024

Real-time and high-precision situational awareness technology is critical for autonomous navigation of unmanned surface vehicles (USVs). In particular, robust and fast obstacle semantic segmentation methods are essential. However, distinguishing between the sea and the sky is challenging due to the differences between port and maritime environments. In this study, we built a dataset that captured perspectives from USVs and unmanned aerial vehicles in a maritime port environment and analysed the data features. Statistical analysis revealed a high correlation between the distribution of the sea and sky and row positional information. Based on this finding, a three-branch semantic segmentation network with a row position encoding module (RPEM) was proposed to improve the prediction accuracy between the sea and the sky. The proposed RPEM highlights the effect of row coordinates on feature extraction. Compared to the baseline, the three-branch network with RPEM significantly improved the ability to distinguish between the sea and the sky without significantly reducing the computational speed.
SpikingJET: Enhancing Fault Injection for Fully and Convolutional Spiking Neural Networks
Venue: arXiv cs.NE 2024
Year: 2024

As artificial neural networks become increasingly integrated into safety-critical systems such as autonomous vehicles, devices for medical diagnosis, and industrial automation, ensuring their reliability in the face of random hardware faults becomes paramount. This paper introduces SpikingJET, a novel fault injector designed specifically for fully connected and convolutional Spiking Neural Networks (SNNs). Our work underscores the critical need to evaluate the resilience of SNNs to hardware faults, considering their growing prominence in real-world applications. SpikingJET provides a comprehensive platform for assessing the resilience of SNNs by inducing errors and injecting faults into critical components such as synaptic weights, neuron model parameters, internal states, and activation functions. This paper demonstrates the effectiveness of Spiking-JET through extensive software-level experiments on various SNN architectures, revealing insights into their vulnerability and resilience to hardware faults. Moreover, highlighting the importance of fault resilience in SNNs contributes to the ongoing effort to enhance the reliability and safety of Neural Network (NN)-powered systems in diverse domains.
Designing a User-centric Framework for Information Quality Ranking of Large-scale Street View Images
Venue: arXiv cs.HC 2024
Year: 2024

Street view imagery (SVI), largely captured via outfitted fleets or mounted dashcams in consumer vehicles is a rapidly growing source of geospatial data used in urban sensing and development. These datasets are often collected opportunistically, are massive in size, and vary in quality which limits the scope and extent of their use in urban planning. Thus far there has not been much work to identify the obstacles experienced and tools needed by the users of such datasets. This severely limits the opportunities of using emerging street view images in supporting novel research questions that can improve the quality of urban life. This work includes a formative interview study with 5 expert users of large-scale street view datasets from academia, urban planning, and related professions which identifies novel use cases, challenges, and opportunities to increase the utility of these datasets. Based on the user findings, we present a framework to evaluate the quality of information for street images across three attributes (spatial, temporal, and content) that stakeholders can utilize for estimating the value of a dataset, and to improve it over time for their respective use case. We then present a case study using novel street view images where we evaluate our framework and present practical use cases for users. We discuss the implications for designing future systems to support the collection and use of street view data to assist in sensing and planning the urban environment.
Value, Representation, Information and Communication
Venue: arXiv cs.IT 2024
Year: 2024

A new analytic framework is first formalized via the usage of the Monadology (Leibniz 1898), to expand the understanding of Zermelo-Fraenkel-choice set theory (ZFC) and Von Neumann-Bernays-Godel set theory (NBG). Implicitly, the framework levels value, representation and information separately. Given the fact that there exists a coincidental equivalence between Von Neumann universe and originally-formalized motivation in ZFC, this work hypothesizes the essential of ordered values for one monand, to carry out efficient communication with the rest. This work then focuses on the relationship among values, representation and information (and suggests potential methods for quantitative analysis). First, this framework generalizes the definition of values and representations from "Indexes approximate Values" principle by (Peng 2023) via surreal numbers (Knuth 1974). Second, credited to surreal numbers, this work recursively connects representations and information via subsets of sets. Therefore, the definition to metric space(s) is naturally formed by representations, and quantitative methods (e.g., Hausdorff Distance) can be applied for quantitative analysis among (sub)sets. Third, this framework conjectures that: as long as the metric space is (or can be formed as) complete, the existence tests can be performed via Cauchy Sequence (or its generalized methods). This work finally revisits the communication theory, and suggests new perspectives from the new analytic framework. Particularly, this work hypothesizes a (quantitative) relationship between values and representation, and conjectures that: the optimal construction of representations exists, and it can be derived as the core value of one monad via Cauchy Inequality (or its generalized methods).
Self-Corrective Sensor Fusion for Drone Positioning in Indoor Facilities
Venue: arXiv cs.RO 2024
Year: 2024

Drones may be more advantageous than fixed cameras for quality control applications in industrial facilities, since they can be redeployed dynamically and adjusted to production planning. The practical scenario that has motivated this paper, image acquisition with drones in a car manufacturing plant, requires drone positioning accuracy in the order of 5 cm. During repetitive manufacturing processes, it is assumed that quality control imaging drones will follow highly deterministic periodic paths, stop at predefined points to take images and send them to image recognition servers. Therefore, by relying on prior knowledge about production chain schedules, it is possible to optimize the positioning technologies for the drones to stay at all times within the boundaries of their flight plans, which will be composed of stopping points and the paths in between. This involves mitigating issues such as temporary blocking of line-of-sight between the drone and any existing radio beacons; sensor data noise; and the loss of visual references. We present a self-corrective solution for this purpose. It corrects visual odometer readings based on filtered and clustered Ultra-Wide Band (UWB) data, as an alternative to direct Kalman fusion. The approach combines the advantages of these technologies when at least one of them works properly at any measurement spot. It has three method components: independent Kalman filtering, data association by means of stream clustering and mutual correction of sensor readings based on the generation of cumulative correction vectors. The approach is inspired by the observation that UWB positioning works reasonably well at static spots whereas visual odometer measurements reflect straight displacements correctly but can underestimate their length. Our experimental results demonstrate the advantages of the approach in the application scenario over Kalman fusion.
Visualizing Routes with AI-Discovered Street-View Patterns
Venue: arXiv cs.HC 2024
Year: 2024

Street-level visual appearances play an important role in studying social systems, such as understanding the built environment, driving routes, and associated social and economic factors. It has not been integrated into a typical geographical visualization interface (e.g., map services) for planning driving routes. In this paper, we study this new visualization task with several new contributions. First, we experiment with a set of AI techniques and propose a solution of using semantic latent vectors for quantifying visual appearance features. Second, we calculate image similarities among a large set of street-view images and then discover spatial imagery patterns. Third, we integrate these discovered patterns into driving route planners with new visualization techniques. Finally, we present VivaRoutes, an interactive visualization prototype, to show how visualizations leveraged with these discovered patterns can help users effectively and interactively explore multiple routes. Furthermore, we conducted a user study to assess the usefulness and utility of VivaRoutes.
Interactive Multi-Robot Flocking with Gesture Responsiveness and Musical Accompaniment
Venue: arXiv cs.RO 2024
Year: 2024

For decades, robotics researchers have pursued various tasks for multi-robot systems, from cooperative manipulation to search and rescue. These tasks are multi-robot extensions of classical robotic tasks and often optimized on dimensions such as speed or efficiency. As robots transition from commercial and research settings into everyday environments, social task aims such as engagement or entertainment become increasingly relevant. This work presents a compelling multi-robot task, in which the main aim is to enthrall and interest. In this task, the goal is for a human to be drawn to move alongside and participate in a dynamic, expressive robot flock. Towards this aim, the research team created algorithms for robot movements and engaging interaction modes such as gestures and sound. The contributions are as follows: (1) a novel group navigation algorithm involving human and robot agents, (2) a gesture responsive algorithm for real-time, human-robot flocking interaction, (3) a weight mode characterization system for modifying flocking behavior, and (4) a method of encoding a choreographer's preferences inside a dynamic, adaptive, learned system. An experiment was performed to understand individual human behavior while interacting with the flock under three conditions: weight modes selected by a human choreographer, a learned model, or subset list. Results from the experiment showed that the perception of the experience was not influenced by the weight mode selection. This work elucidates how differing task aims such as engagement manifest in multi-robot system design and execution, and broadens the domain of multi-robot tasks.
Synthetic Dataset Generation and Learning From Demonstration Applied to Industrial Manipulation
Venue: arXiv cs.RO 2024
Year: 2024

The aim of this study is to investigate an automated industrial manipulation pipeline, where assembly tasks can be flexibly adapted to production without the need for a robotic expert, both for the vision system and the robot program. The objective of this study is first, to develop a synthetic-dataset-generation pipeline with a special focus on industrial parts, and second, to use Learning-from-Demonstration (LfD) methods to replace manual robot programming, so that a non-robotic expert/process engineer can introduce a new manipulation task by teaching it to the robot.
MetaIE: Distilling a Meta Model from LLM for All Kinds of Information Extraction Tasks
Venue: arXiv cs.CL 2024
Year: 2024

Information extraction (IE) is a fundamental area in natural language processing where prompting large language models (LLMs), even with in-context examples, cannot defeat small LMs tuned on very small IE datasets. We observe that IE tasks, such as named entity recognition and relation extraction, all focus on extracting important information, which can be formalized as a label-to-span matching. In this paper, we propose a novel framework MetaIE to build a small LM as meta-model by learning to extract "important information", i.e., the meta-understanding of IE, so that this meta-model can be adapted to all kind of IE tasks effectively and efficiently. Specifically, MetaIE obtains the small LM via a symbolic distillation from an LLM following the label-to-span scheme. We construct the distillation dataset via sampling sentences from language model pre-training datasets (e.g., OpenWebText in our implementation) and prompting an LLM to identify the typed spans of "important information". We evaluate the meta-model under the few-shot adaptation setting. Extensive results on 13 datasets from 6 IE tasks confirm that MetaIE can offer a better starting point for few-shot tuning on IE datasets and outperform other meta-models from (1) vanilla language model pre-training, (2) multi-IE-task pre-training with human annotations, and (3) single-IE-task symbolic distillation from LLM. Moreover, we provide comprehensive analyses of MetaIE, such as the size of the distillation dataset, the meta-model architecture, and the size of the meta-model.
Competition-Aware Decision-Making Approach for Mobile Robots in Racing Scenarios
Venue: arXiv cs.RO 2024
Year: 2024

This paper presents a game-theoretic strategy for racing, where the autonomous ego agent seeks to block a racing opponent that aims to overtake the ego agent. After a library of trajectory candidates and an associated reward matrix are constructed, the optimal trajectory in terms of maximizing the cumulative reward over the planning horizon is determined based on the level-K reasoning framework. In particular, the level of the opponent is estimated online according to its behavior over a past window and is then used to determine the trajectory for the ego agent. Taking into account that the opponent may change its level and strategy during the decision process of the ego agent, we introduce a trajectory mixing strategy that blends the level-K optimal trajectory with a fail-safe trajectory. The overall algorithm was tested and evaluated in various simulated racing scenarios, which also includes human-in-the-loop experiments. Comparative analysis against the conventional level-K framework demonstrates the superiority of our proposed approach in terms of overtake-blocking success rates.
Super Non-singular Decompositions of Polynomials and their Application to Robustly Learning Low-degree PTFs
Venue: arXiv cs.DS 2024
Year: 2024

We study the efficient learnability of low-degree polynomial threshold functions (PTFs) in the presence of a constant fraction of adversarial corruptions. Our main algorithmic result is a polynomial-time PAC learning algorithm for this concept class in the strong contamination model under the Gaussian distribution with error guarantee $O_{d, c}(\text{opt}^{1-c})$, for any desired constant $c>0$, where $\text{opt}$ is the fraction of corruptions. In the strong contamination model, an omniscient adversary can arbitrarily corrupt an $\text{opt}$-fraction of the data points and their labels. This model generalizes the malicious noise model and the adversarial label noise model. Prior to our work, known polynomial-time algorithms in this corruption model (or even in the weaker adversarial label noise model) achieved error $\tilde{O}_d(\text{opt}^{1/(d+1)})$, which deteriorates significantly as a function of the degree $d$. Our algorithm employs an iterative approach inspired by localization techniques previously used in the context of learning linear threshold functions. Specifically, we use a robust perceptron algorithm to compute a good partial classifier and then iterate on the unclassified points. In order to achieve this, we need to take a set defined by a number of polynomial inequalities and partition it into several well-behaved subsets. To this end, we develop new polynomial decomposition techniques that may be of independent interest.
ADs: Active Data-sharing for Data Quality Assurance in Advanced Manufacturing Systems
Venue: arXiv cs.LG 2024
Year: 2024

Machine learning (ML) methods are widely used in industrial applications, which usually require a large amount of training data. However, data collection needs extensive time costs and investments in the manufacturing system, and data scarcity commonly exists. Therefore, data-sharing is widely enabled among multiple machines with similar functionality to augment the dataset for building ML methods. However, distribution mismatch inevitably exists in their data due to different working conditions, while the ML methods are assumed to be built and tested on the dataset following the same distribution. Thus, an Active Data-sharing (ADs) framework is proposed to ensure the quality of the shared data among multiple machines. It is designed to simultaneously select the most informative data points benefiting the downstream tasks and mitigate the distribution mismatch among all selected data points. The proposed method is validated on anomaly detection on in-situ monitoring data from three additive manufacturing processes.
Automated Bi-Fold Weighted Ensemble Algorithms and its Application to Brain Tumor Detection and Classification
Venue: arXiv cs.LG 2024
Year: 2024

The uncontrolled and unstructured growth of brain cells is known as brain tumor, which has one of the highest mortality rates among diseases from all types of cancers. Due to limited diagnostic and treatment capabilities, they pose significant challenges, especially in third-world countries. Early diagnosis plays a vital role in effectively managing brain tumors and reducing mortality rates. However, the availability of diagnostic methods is hindered by various limitations, including high costs and lengthy result acquisition times, impeding early detection of the disease. In this study, we present two cutting-edge bi-fold weighted voting ensemble models that aim to boost the effectiveness of weighted ensemble methods. These two proposed methods combine the classification outcomes from multiple classifiers and determine the optimal result by selecting the one with the highest probability in the first approach, and the highest weighted prediction in the second technique. These approaches significantly improve the overall performance of weighted ensemble techniques. In the first proposed method, we improve the soft voting technique (SVT) by introducing a novel unsupervised weight calculating schema (UWCS) to enhance its weight assigning capability, known as the extended soft voting technique (ESVT). Secondly, we propose a novel weighted method (NWM) by using the proposed UWCS. Both of our approaches incorporate three distinct models: a custom-built CNN, VGG-16, and InceptionResNetV2 which has been trained on publicly available datasets. The effectiveness of our proposed systems is evaluated through blind testing, where exceptional results are achieved. We then establish a comparative analysis of the performance of our proposed methods with that of SVT to show their superiority and effectiveness.
EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositories
Venue: arXiv cs.CL 2024
Year: 2024

How to evaluate Large Language Models (LLMs) in code generation is an open question. Existing benchmarks demonstrate poor alignment with real-world code repositories and are insufficient to evaluate the coding abilities of LLMs. This paper proposes a new benchmark - EvoCodeBench to address the preceding problems, which has three primary advances. (1) EvoCodeBench aligns with real-world repositories in multiple dimensions, e.g., code distributions and dependency distributions. (2) EvoCodeBench offers comprehensive annotations (e.g., requirements, reference code, and reference dependencies), and robust evaluation metrics (e.g., Pass@k and Recall@k). (3) EvoCodeBench is an evolving benchmark to avoid data leakage. We build an automatic pipeline to update EvoCodeBench from the latest repositories. We release the first version - EvoCodeBench-2403, containing 275 samples from 25 real-world repositories. Based on EvoCodeBench, we propose repository-level code generation and evaluate 10 popular LLMs (e.g., gpt-4, gpt-3.5, DeepSeek Coder, StarCoder 2, CodeLLaMa, Gemma, and Qwen 1.5). Our experiments reveal the coding abilities of these LLMs in real-world repositories. For example, the highest Pass@1 of gpt-4 only is 20.73% in our experiments. We also analyze failed cases and summarize the shortcomings of existing LLMs in EvoCodeBench. We release EvoCodeBench, all prompts, and LLMs' completions for further community analysis.
Extensive Self-Contrast Enables Feedback-Free Language Model Alignment
Venue: arXiv cs.CL 2024
Year: 2024

Reinforcement learning from human feedback (RLHF) has been a central technique for recent large language model (LLM) alignment. However, its heavy dependence on costly human or LLM-as-Judge preference feedback could stymie its wider applications. In this work, we introduce Self-Contrast, a feedback-free large language model alignment method via exploiting extensive self-generated negatives. With only supervised fine-tuning (SFT) targets, Self-Contrast leverages the LLM itself to generate massive diverse candidates, and harnesses a pre-trained embedding model to filter multiple negatives according to text similarity. Theoretically, we illustrate that in this setting, merely scaling negative responses can still effectively approximate situations with more balanced positive and negative preference annotations. Our experiments with direct preference optimization (DPO) on three datasets show that, Self-Contrast could consistently outperform SFT and standard DPO training by large margins. And as the number of self-generated negatives increases, the performance of Self-Contrast continues to grow. Code and data are available at https://github.com/THUDM/Self-Contrast.