OpenRaiser/Intern-Atlas huggingface dataset:
Test-Time Training for Semantic Segmentation with Output Contrastive Loss
Venue: arXiv cs.CV 2023
Year: 2023

Although deep learning-based segmentation models have achieved impressive performance on public benchmarks, generalizing well to unseen environments remains a major challenge. To improve the model's generalization ability to the new domain during evaluation, the test-time training (TTT) is a challenging paradigm that adapts the source-pretrained model in an online fashion. Early efforts on TTT mainly focus on the image classification task. Directly extending these methods to semantic segmentation easily experiences unstable adaption due to segmentation's inherent characteristics, such as extreme class imbalance and complex decision spaces. To stabilize the adaptation process, we introduce contrastive loss (CL), known for its capability to learn robust and generalized representations. Nevertheless, the traditional CL operates in the representation space and cannot directly enhance predictions. In this paper, we resolve this limitation by adapting the CL to the output space, employing a high temperature, and simplifying the formulation, resulting in a straightforward yet effective loss function called Output Contrastive Loss (OCL). Our comprehensive experiments validate the efficacy of our approach across diverse evaluation scenarios. Notably, our method excels even when applied to models initially pre-trained using domain adaptation methods on test domain data, showcasing its resilience and adaptability.\footnote{Code and more information could be found at~ \url{https://github.com/dazhangyu123/OCL}}
One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion
Venue: arXiv cs.CV 2023
Year: 2023

Recent advancements in open-world 3D object generation have been remarkable, with image-to-3D methods offering superior fine-grained control over their text-to-3D counterparts. However, most existing models fall short in simultaneously providing rapid generation speeds and high fidelity to input images - two features essential for practical applications. In this paper, we present One-2-3-45++, an innovative method that transforms a single image into a detailed 3D textured mesh in approximately one minute. Our approach aims to fully harness the extensive knowledge embedded in 2D diffusion models and priors from valuable yet limited 3D data. This is achieved by initially finetuning a 2D diffusion model for consistent multi-view image generation, followed by elevating these images to 3D with the aid of multi-view conditioned 3D native diffusion models. Extensive experimental evaluations demonstrate that our method can produce high-quality, diverse 3D assets that closely mirror the original input image. Our project webpage: https://sudo-ai-3d.github.io/One2345plus_page.
Challenges of Securing Massively Multiplayer Online Games
Venue: arXiv cs.CR 2023
Year: 2023

When it comes to security in the modern world, things have improved a lot since the early 2000s. Hypertext Transfer Protocol Secure (HTTPS) and Transport Layer Security (TLS) have made the transfer of our data across the internet much safer than years prior, and the advent of VPNs and private browsing have only compounded that. However, the gaming industry has been notoriously behind the curve when it comes to security, most notably with Massively Multiplayer Online (MMO) games, which due to the intrinsic nature of their architecture, have an astounding amount of ground to cover. In this paper, the authors discuss the challenges that MMO developers face when trying to design a secure game, as well as some more modern approaches to security that will help improve the industry moving forward. The authors also highlight a few real-life examples of exploits and breaches that have happened and look at how they were mitigated.
CPopQA: Ranking Cultural Concept Popularity by LLMs
Venue: arXiv cs.CL 2023
Year: 2023

Prior work has demonstrated large language models' (LLMs) potential to discern statistical tendencies within their pre-training corpora. Despite that, many examinations of LLMs' knowledge capacity focus on knowledge explicitly appearing in the training data or implicitly inferable from similar contexts. How well an LLM captures the corpus-level statistical trends of concepts for reasoning, especially long-tail ones, is still underexplored. In this study, we introduce a novel few-shot question-answering task (CPopQA) that examines LLMs' statistical ranking abilities for long-tail cultural concepts (e.g., holidays), with a specific focus on these concepts' popularity in the United States and the United Kingdom, respectively. We curate a dataset containing 459 holidays across 58 countries, generating a total of 6,000 QA testing pairs. Experiments on four strong LLMs show that large models are capable of ranking long-tail cultural concepts regarding their statistical tendency. Notably, GPT-3.5 displayed superior performance and exhibited its potential to identify geo-cultural proximity across continents.
Detection of Small Targets in Sea Clutter Based on RepVGG and Continuous Wavelet Transform
Venue: arXiv cs.CV 2023
Year: 2023

Constructing a high-performance target detector under the background of sea clutter is always necessary and important. In this work, we propose a RepVGGA0-CWT detector, where RepVGG is a residual network that gains a high detection accuracy. Different from traditional residual networks, RepVGG keeps an acceptable calculation speed. Giving consideration to both accuracy and speed, the RepVGGA0 is selected among all the variants of RepVGG. Also, continuous wavelet transform (CWT) is employed to extract the radar echoes' time-frequency feature effectively. In the tests, other networks (ResNet50, ResNet18 and AlexNet) and feature extraction methods (short-time Fourier transform (STFT), CWT) are combined to build detectors for comparison. The result of different datasets shows that the RepVGGA0-CWT detector performs better than those detectors in terms of low controllable false alarm rate, high training speed, high inference speed and low memory usage. This RepVGGA0-CWT detector is hardware-friendly and can be applied in real-time scenes for its high inference speed in detection.
bpftime: userspace eBPF Runtime for Uprobe, Syscall and Kernel-User Interactions
Venue: arXiv cs.OS 2023
Year: 2023

In kernel-centric operations, the uprobe component of eBPF frequently encounters performance bottlenecks, largely attributed to the overheads borne by context switches. Transitioning eBPF operations to user space bypasses these hindrances, thereby optimizing performance. This also enhances configurability and obviates the necessity for root access or privileges for kernel eBPF, subsequently minimizing the kernel attack surface. This paper introduces bpftime, a novel user-space eBPF runtime, which leverages binary rewriting to implement uprobe and syscall hook capabilities. Through bpftime, userspace uprobes achieve a 10x speed enhancement compared to their kernel counterparts without requiring dual context switches. Additionally, this runtime facilitates the programmatic hooking of syscalls within a process, both safely and efficiently. Bpftime can be seamlessly attached to any running process, limiting the need for either a restart or manual recompilation. Our implementation also extends to interprocess eBPF Maps within shared memory, catering to summary aggregation or control plane communication requirements. Compatibility with existing eBPF toolchains such as clang and libbpf is maintained, not only simplifying the development of user-space eBPF without necessitating any modifications but also supporting CO-RE through BTF. Through bpftime, we not only enhance uprobe performance but also extend the versatility and user-friendliness of eBPF runtime in user space, paving the way for more efficient and secure kernel operations.
Brain-Driven Representation Learning Based on Diffusion Model
Venue: arXiv cs.CL 2023
Year: 2023

Interpreting EEG signals linked to spoken language presents a complex challenge, given the data's intricate temporal and spatial attributes, as well as the various noise factors. Denoising diffusion probabilistic models (DDPMs), which have recently gained prominence in diverse areas for their capabilities in representation learning, are explored in our research as a means to address this issue. Using DDPMs in conjunction with a conditional autoencoder, our new approach considerably outperforms traditional machine learning algorithms and established baseline models in accuracy. Our results highlight the potential of DDPMs as a sophisticated computational method for the analysis of speech-related EEG signals. This could lead to significant advances in brain-computer interfaces tailored for spoken communication.
VizPut: Insight-Aware Imputation of Incomplete Data for Visualization Recommendation
Venue: arXiv cs.DB 2023
Year: 2023

In insight recommendation systems, obtaining timely and high-quality recommended visual analytics over incomplete data is challenging due to the difficulties in cleaning and processing such data. Failing to address data incompleteness results in diminished recommendation quality, compelling users to impute the incomplete data to a cleaned version through a costly imputation strategy. This paper introduces VizPut scheme, an insight-aware selective imputation technique capable of determining which missing values should be imputed in incomplete data to optimize the effectiveness of recommended visualizations within a specified imputation budget. The VizPut scheme determines the optimal allocation of imputation operations with the objective of achieving maximal effectiveness in recommended visual analytics. We evaluate this approach using real-world datasets, and our experimental results demonstrate that VizPut effectively maximizes the efficacy of recommended visualizations within the user-defined imputation budget.
Surveying Wikipedians: a dataset of users and contributors' practices on Wikipedia in 8 languages
Venue: arXiv cs.CY 2023
Year: 2023

The dataset focuses on Wikipedia users and contains information about demographic and socioeconomic characteristics of the respondents and their activity on Wikipedia. The data was collected using a questionnaire available online between June and July 2023. The link to the questionnaire was distributed via a banner published in 8 languages on the Wikipedia page. Filling out the questionnaire was voluntary and not incentivised in any way. The survey includes 200 questions about: what people were doing on Wikipedia before clicking the link to the questionnaire; how they use Wikipedia as readers (``professional'' and ``personal'' uses); their opinion on the quality, the thematic coverage, the importance of the encyclopaedia; the making of Wikipedia (how they think it is made, if they have ever contributed and how); their social, sport, artistic and cultural activities, both online and offline; their socio-economic characteristics including political beliefs, and trust propensities. More than 200 000 people opened the questionnaire, 100 332 started to answer, and constitute our dataset, and 10 576 finished it. Among other themes identified by future researchers, the dataset can be useful for advancing the research regarding the features of readers vs contributors of online commons, the relationship between trust, information, sources, and the use made of this information.
Benchmarking Individual Tree Mapping with Sub-meter Imagery
Venue: arXiv cs.CV 2023
Year: 2023

There is a rising interest in mapping trees using satellite or aerial imagery, but there is no standardized evaluation protocol for comparing and enhancing methods. In dense canopy areas, the high variability of tree sizes and their spatial proximity makes it arduous to define the quality of the predictions. Concurrently, object-centric approaches such as bounding box detection usuallyperform poorly on small and dense objects. It thus remains unclear what is the ideal framework for individual tree mapping, in regards to detection and segmentation approaches, convolutional neural networks and transformers. In this paper, we introduce an evaluation framework suited for individual tree mapping in any physical environment, with annotation costs and applicative goals in mind. We review and compare different approaches and deep architectures, and introduce a new method that we experimentally prove to be a good compromise between segmentation and detection.
Configurable convolutional neural networks for real-time pedestrian-level wind prediction in urban environments
Venue: arXiv cs.CE 2023
Year: 2023

Urbanization has underscored the importance of understanding the pedestrian wind environment in urban and architectural design contexts. Pedestrian Wind Comfort (PWC) focuses on the effects of wind on the safety and comfort of pedestrians and cyclists, given the influence of urban structures on the local microclimate. Traditional Computational Fluid Dynamics (CFD) methods used for PWC analysis have limitations in computation, cost, and time. Deep-learning models have the potential to significantly speed up this process. The prevailing state-of-the-art methodologies largely rely on GAN-based models, such as pix2pix, which have exhibited training instability issues. In contrast, our work introduces a convolutional neural network (CNN) approach based on the U-Net architecture, offering a more stable and streamlined solution. The process of generating a wind flow prediction at pedestrian level is reformulated from a 3D CFD simulation into a 2D image-to-image translation task, using the projected building heights as input. Testing on standard consumer hardware shows that our model can efficiently predict wind velocities in urban settings in real time. Further tests on different configurations of the model, combined with a Pareto front analysis, helped identify the trade-off between accuracy and computational efficiency. This CNN-based approach provides a fast and efficient method for PWC analysis, potentially aiding in more efficient urban design processes.
Quantum Algorithms for Graph Coloring and other Partitioning, Covering, and Packing Problems
Venue: arXiv cs.DS 2023
Year: 2023

Let U be a universe on n elements, let k be a positive integer, and let F be a family of (implicitly defined) subsets of U. We consider the problems of partitioning U into k sets from F, covering U with k sets from F, and packing k non-intersecting sets from F into U. Classically, these problems can be solved via inclusion-exclusion in O*(2^n) time [BjorklundHK09]. Quantumly, there are faster algorithms for graph coloring with running time O(1.9140^n) [ShimizuM22] and for Set Cover with a small number of sets with running time O(1.7274^n |F|^O(1)) [AmbainisBIKPV19]. In this paper, we give a quantum speedup for Set Partition, Set Cover, and Set Packing whenever there is a classical enumeration algorithm that lends itself to a quadratic quantum speedup, which, for any subinstance on a subset X of U, enumerates at least one member of a k-partition, k-cover, or k-packing (if one exists) restricted to (or projected onto, in the case of k-cover) the set X in O*(c^{|X|}) time with c<2. Our bounded-error quantum algorithm runs in O*((2+c)^(n/2)) for Set Partition, Set Cover, and Set Packing. When c<=1.147899, our algorithm is slightly faster than O*((2+c)^(n/2)); when c approaches 1, it matches the running time of [AmbainisBIKPV19] for Set Cover when |F| is subexponential in n. For Graph Coloring, we further improve the running time to O(1.7956^n) by leveraging faster algorithms for coloring with a small number of colors to better balance our divide-and-conquer steps. For Domatic Number, we obtain a O((2-ε)^n) running time for some ε>0.
PASDA: A Partition-based Semantic Differencing Approach with Best Effort Classification of Undecided Cases
Venue: arXiv cs.SE 2023
Year: 2023

Equivalence checking is used to verify whether two programs produce equivalent outputs when given equivalent inputs. Research in this field mainly focused on improving equivalence checking accuracy and runtime performance. However, for program pairs that cannot be proven to be either equivalent or non-equivalent, existing approaches only report a classification result of "unknown", which provides no information regarding the programs' non-/equivalence. In this paper, we introduce PASDA, our partition-based semantic differencing approach with best effort classification of undecided cases. While PASDA aims to formally prove non-/equivalence of analyzed program pairs using a variant of differential symbolic execution, its main novelty lies in its handling of cases for which no formal non-/equivalence proof can be found. For such cases, PASDA provides a best effort equivalence classification based on a set of classification heuristics. We evaluated PASDA with an existing benchmark consisting of 141 non-/equivalent program pairs. PASDA correctly classified 61-74% of these cases at timeouts from 10 seconds to 3600 seconds. Thus, PASDA achieved equivalence checking accuracies that are 3-7% higher than the best results achieved by three existing tools. Furthermore, PASDA's best effort classifications were correct for 70-75% of equivalent and 55-85% of non-equivalent cases across the different timeouts.
CPSOR-GCN: A Vehicle Trajectory Prediction Method Powered by Emotion and Cognitive Theory
Venue: arXiv cs.AI 2023
Year: 2023

Active safety systems on vehicles often face problems with false alarms. Most active safety systems predict the driver's trajectory with the assumption that the driver is always in a normal emotion, and then infer risks. However, the driver's trajectory uncertainty increases under abnormal emotions. This paper proposes a new trajectory prediction model: CPSOR-GCN, which predicts vehicle trajectories under abnormal emotions. At the physical level, the interaction features between vehicles are extracted by the physical GCN module. At the cognitive level, SOR cognitive theory is used as prior knowledge to build a Dynamic Bayesian Network (DBN) structure. The conditional probability and state transition probability of nodes from the calibrated SOR-DBN quantify the causal relationship between cognitive factors, which is embedded into the cognitive GCN module to extract the characteristics of the influence mechanism of emotions on driving behavior. The CARLA-SUMO joint driving simulation platform was built to develop dangerous pre-crash scenarios. Methods of recreating traffic scenes were used to naturally induce abnormal emotions. The experiment collected data from 26 participants to verify the proposed model. Compared with the model that only considers physical motion features, the prediction accuracy of the proposed model is increased by 68.70%. Furthermore,considering the SOR-DBN reduces the prediction error of the trajectory by 15.93%. Compared with other advanced trajectory prediction models, the results of CPSOR-GCN also have lower errors. This model can be integrated into active safety systems to better adapt to the driver's emotions, which could effectively reduce false alarms.
Channel Estimation with Dynamic Metasurface Antennas via Model-Based Learning
Venue: arXiv cs.IT 2023
Year: 2023

Dynamic Metasurface Antenna (DMA) is a cutting-edge antenna technology offering scalable and sustainable solutions for large antenna arrays. The effectiveness of DMAs stems from their inherent configurable analog signal processing capabilities, which facilitate cost-limited implementations. However, when DMAs are used in multiple input multiple output (MIMO) communication systems, they pose challenges in channel estimation due to their analog compression. In this paper, we propose two model-based learning methods to overcome this challenge. Our approach starts by casting channel estimation as a compressed sensing problem. Here, the sensing matrix is formed using a random DMA weighting matrix combined with a spatial gridding dictionary. We then employ the learned iterative shrinkage and thresholding algorithm (LISTA) to recover the sparse channel parameters. LISTA unfolds the iterative shrinkage and thresholding algorithm into a neural network and trains the neural network into a highly efficient channel estimator fitting with the previous channel. As the sensing matrix is crucial to the accuracy of LISTA recovery, we introduce another data-aided method, LISTA-sensing matrix optimization (LISTA-SMO), to jointly optimize the sensing matrix. LISTA-SMO takes LISTA as a backbone and embeds the sensing matrix optimization layers in LISTA's neural network, allowing for the optimization of the sensing matrix along with the training of LISTA. Furthermore, we propose a self-supervised learning technique to tackle the difficulty of acquiring noise-free data. Our numerical results demonstrate that LISTA outperforms traditional sparse recovery methods regarding channel estimation accuracy and efficiency. Besides, LISTA-SMO achieves better channel accuracy than LISTA, demonstrating the effectiveness in optimizing the sensing matrix.
DynamicSurf: Dynamic Neural RGB-D Surface Reconstruction with an Optimizable Feature Grid
Venue: arXiv cs.CV 2023
Year: 2023

We propose DynamicSurf, a model-free neural implicit surface reconstruction method for high-fidelity 3D modelling of non-rigid surfaces from monocular RGB-D video. To cope with the lack of multi-view cues in monocular sequences of deforming surfaces, one of the most challenging settings for 3D reconstruction, DynamicSurf exploits depth, surface normals, and RGB losses to improve reconstruction fidelity and optimisation time. DynamicSurf learns a neural deformation field that maps a canonical representation of the surface geometry to the current frame. We depart from current neural non-rigid surface reconstruction models by designing the canonical representation as a learned feature grid which leads to faster and more accurate surface reconstruction than competing approaches that use a single MLP. We demonstrate DynamicSurf on public datasets and show that it can optimize sequences of varying frames with $6\times$ speedup over pure MLP-based approaches while achieving comparable results to the state-of-the-art methods. Project is available at https://mirgahney.github.io//DynamicSurf.io/.
All Data on the Table: Novel Dataset and Benchmark for Cross-Modality Scientific Information Extraction
Venue: arXiv cs.CL 2023
Year: 2023

Extracting key information from scientific papers has the potential to help researchers work more efficiently and accelerate the pace of scientific progress. Over the last few years, research on Scientific Information Extraction (SciIE) witnessed the release of several new systems and benchmarks. However, existing paper-focused datasets mostly focus only on specific parts of a manuscript (e.g., abstracts) and are single-modality (i.e., text- or table-only), due to complex processing and expensive annotations. Moreover, core information can be present in either text or tables or across both. To close this gap in data availability and enable cross-modality IE, while alleviating labeling costs, we propose a semi-supervised pipeline for annotating entities in text, as well as entities and relations in tables, in an iterative procedure. Based on this pipeline, we release novel resources for the scientific community, including a high-quality benchmark, a large-scale corpus, and a semi-supervised annotation pipeline. We further report the performance of state-of-the-art IE models on the proposed benchmark dataset, as a baseline. Lastly, we explore the potential capability of large language models such as ChatGPT for the current task. Our new dataset, results, and analysis validate the effectiveness and efficiency of our semi-supervised pipeline, and we discuss its remaining limitations.