OpenRaiser/Intern-Atlas huggingface dataset:
Towards scalable cryogenic quantum dot biasing using memristor-based DC sources
Venue: arXiv cs.ET 2024
Year: 2024

Cryogenic memristor-based DC sources offer a promising avenue for in situ biasing of quantum dot arrays. In this study, we present experimental results and discuss the scaling potential for such DC sources. We first demonstrate the operation of a commercial discrete operational amplifier down to 1.2K which is used on the DC source prototype. Then, the tunability of the memristor-based DC source is validated by performing several 250mV-DC sweeps with a resolution of 10mV at room temperature and at 1.2K. Additionally, the DC source prototype exhibits a limited output drift of $\approx1\mathrm{μVs^{-1}}$ at 1.2K. This showcases the potential of memristor-based DC sources for quantum dot biasing. Limitations in power consumption and voltage resolution using discrete components highlight the need for a fully integrated and scalable complementary metal-oxide-semiconductor-based (CMOS-based) approach. To address this, we propose to monolithically co-integrate emerging non-volatile memories (eNVMs) and 65nm CMOS circuitry. Simulations reveal a reduction in power consumption, down to $\mathrm{10μW}$ per DC source and in footprint. This allows for the integration of up to one million eNVM-based DC sources at the 4.2K stage of a dilution fridge, paving the way for near term large-scale quantum computing applications.
Tetris with Few Piece Types
Venue: arXiv cs.CC 2024
Year: 2024

We prove NP-hardness and #P-hardness of Tetris clearing (clearing an initial board using a given sequence of pieces) with the Super Rotation System (SRS), even when the pieces are limited to any two of the seven Tetris piece types. This result is the first advance on a question posed twenty years ago: which piece sets are easy vs. hard? All previous Tetris NP-hardness proofs used five of the seven piece types. We also prove ASP-completeness of Tetris clearing, using three piece types, as well as versions of 3-Partition and Numerical 3-Dimensional Matching where all input integers are distinct. Finally, we prove NP-hardness of Tetris survival and clearing under the "hard drops only" and "20G" modes, using two piece types, improving on a previous "hard drops only" result that used five piece types.
What is Meant by AGI? On the Definition of Artificial General Intelligence
Venue: arXiv cs.AI 2024
Year: 2024

This paper aims to establish a consensus on AGI's definition. General intelligence refers to the adaptation to open environments according to certain principles using limited resources. It emphasizes that adaptation or learning is an indispensable property of intelligence, and places the controversial part within the principles of intelligence, which can be described from different perspectives.
Watch Your Step: Optimal Retrieval for Continual Learning at Scale
Venue: arXiv cs.CV 2024
Year: 2024

In continual learning, a model learns incrementally over time while minimizing interference between old and new tasks. One of the most widely used approaches in continual learning is referred to as replay. Replay methods support interleaved learning by storing past experiences in a replay buffer. Although there are methods for selectively constructing the buffer and reprocessing its contents, there is limited exploration of the problem of selectively retrieving samples from the buffer. Current solutions have been tested in limited settings and, more importantly, in isolation. Existing work has also not explored the impact of duplicate replays on performance. In this work, we propose a framework for evaluating selective retrieval strategies, categorized by simple, independent class- and sample-selective primitives. We evaluated several combinations of existing strategies for selective retrieval and present their performances. Furthermore, we propose a set of strategies to prevent duplicate replays and explore whether new samples with low loss values can be learned without replay. In an effort to match our problem setting to a realistic continual learning pipeline, we restrict our experiments to a setting involving a large, pre-trained, open vocabulary object detection model, which is fully fine-tuned on a sequence of 15 datasets.
Learning Feature Inversion for Multi-class Anomaly Detection under General-purpose COCO-AD Benchmark
Venue: arXiv cs.CV 2024
Year: 2024

Anomaly detection (AD) is often focused on detecting anomaly areas for industrial quality inspection and medical lesion examination. However, due to the specific scenario targets, the data scale for AD is relatively small, and evaluation metrics are still deficient compared to classic vision tasks, such as object detection and semantic segmentation. To fill these gaps, this work first constructs a large-scale and general-purpose COCO-AD dataset by extending COCO to the AD field. This enables fair evaluation and sustainable development for different methods on this challenging benchmark. Moreover, current metrics such as AU-ROC have nearly reached saturation on simple datasets, which prevents a comprehensive evaluation of different methods. Inspired by the metrics in the segmentation field, we further propose several more practical threshold-dependent AD-specific metrics, ie, m$F_1$$^{.2}_{.8}$, mAcc$^{.2}_{.8}$, mIoU$^{.2}_{.8}$, and mIoU-max. Motivated by GAN inversion's high-quality reconstruction capability, we propose a simple but more powerful InvAD framework to achieve high-quality feature reconstruction. Our method improves the effectiveness of reconstruction-based methods on popular MVTec AD, VisA, and our newly proposed COCO-AD datasets under a multi-class unsupervised setting, where only a single detection model is trained to detect anomalies from different classes. Extensive ablation experiments have demonstrated the effectiveness of each component of our InvAD. Full codes and models are available at https://github.com/zhangzjn/ader.
Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes
Venue: arXiv cs.CV 2024
Year: 2024

Recently, 3D Gaussian Splatting (3DGS) has demonstrated impressive novel view synthesis results, while allowing the rendering of high-resolution images in real-time. However, leveraging 3D Gaussians for surface reconstruction poses significant challenges due to the explicit and disconnected nature of 3D Gaussians. In this work, we present Gaussian Opacity Fields (GOF), a novel approach for efficient, high-quality, and adaptive surface reconstruction in unbounded scenes. Our GOF is derived from ray-tracing-based volume rendering of 3D Gaussians, enabling direct geometry extraction from 3D Gaussians by identifying its levelset, without resorting to Poisson reconstruction or TSDF fusion as in previous work. We approximate the surface normal of Gaussians as the normal of the ray-Gaussian intersection plane, enabling the application of regularization that significantly enhances geometry. Furthermore, we develop an efficient geometry extraction method utilizing Marching Tetrahedra, where the tetrahedral grids are induced from 3D Gaussians and thus adapt to the scene's complexity. Our evaluations reveal that GOF surpasses existing 3DGS-based methods in surface reconstruction and novel view synthesis. Further, it compares favorably to or even outperforms, neural implicit methods in both quality and speed.
Reconfigurable Edge Hardware for Intelligent IDS: Systematic Approach
Venue: arXiv cs.CR 2024
Year: 2024

Intrusion detection systems (IDS) are crucial security measures nowadays to enforce network security. Their task is to detect anomalies in network communication and identify, if not thwart, possibly malicious behavior. Recently, machine learning has been deployed to construct intelligent IDS. This approach, however, is quite challenging particularly in distributed, highly dynamic, yet resource-constrained systems like Edge setups. In this paper, we tackle this issue from multiple angles by analyzing the concept of intelligent IDS (I-IDS) while addressing the specific requirements of Edge devices with a special focus on reconfigurability. Then, we introduce a systematic approach to constructing the I-IDS on reconfigurable Edge hardware. For this, we implemented our proposed IDS on state-of-the-art Field Programmable Gate Arrays (FPGAs) technology as (1) a purely FPGA-based dataflow processor (DFP) and (2) a co-designed approach featuring RISC-V soft-core as FPGA-based soft-core processor (SCP). We complete our paper with a comparison of the state of the art (SoA) in this domain. The results show that DFP and SCP are both suitable for Edge applications from hardware resource and energy efficiency perspectives. Our proposed DFP solution clearly outperforms the SoA and demonstrates that required high performance can be achieved without prohibitively high hardware costs. This makes our proposed DFP suitable for Edge-based high-speed applications like modern communication technology.
Integrating Graph Neural Networks with Scattering Transform for Anomaly Detection
Venue: arXiv cs.CR 2024
Year: 2024

In this paper, we present two novel methods in Network Intrusion Detection Systems (NIDS) using Graph Neural Networks (GNNs). The first approach, Scattering Transform with E-GraphSAGE (STEG), utilizes the scattering transform to conduct multi-resolution analysis of edge feature vectors. This provides a detailed representation that is essential for identifying subtle anomalies in network traffic. The second approach improves node representation by initiating with Node2Vec, diverging from standard methods of using uniform values, thereby capturing a more accurate and holistic network picture. Our methods have shown significant improvements in performance compared to existing state-of-the-art methods in benchmark NIDS datasets.
A LayoutLMv3-Based Model for Enhanced Relation Extraction in Visually-Rich Documents
Venue: arXiv cs.CL 2024
Year: 2024

Document Understanding is an evolving field in Natural Language Processing (NLP). In particular, visual and spatial features are essential in addition to the raw text itself and hence, several multimodal models were developed in the field of Visual Document Understanding (VDU). However, while research is mainly focused on Key Information Extraction (KIE), Relation Extraction (RE) between identified entities is still under-studied. For instance, RE is crucial to regroup entities or obtain a comprehensive hierarchy of data in a document. In this paper, we present a model that, initialized from LayoutLMv3, can match or outperform the current state-of-the-art results in RE applied to Visually-Rich Documents (VRD) on FUNSD and CORD datasets, without any specific pre-training and with fewer parameters. We also report an extensive ablation study performed on FUNSD, highlighting the great impact of certain features and modelization choices on the performances.
UruDendro, a public dataset of cross-section images of Pinus taeda
Venue: arXiv cs.CV 2024
Year: 2024

The automatic detection of tree-ring boundaries and other anatomical features using image analysis has progressed substantially over the past decade with advances in machine learning and imagery technology, as well as increasing demands from the dendrochronology community. This paper presents a publicly available database of 64 scanned images of transverse sections of commercially grown Pinus taeda trees from northern Uruguay, ranging from 17 to 24 years old. The collection contains several challenging features for automatic ring detection, including illumination and surface preparation variation, fungal infection (blue stains), knot formation, missing cortex or interruptions in outer rings, and radial cracking. This dataset can be used to develop and test automatic tree ring detection algorithms. This paper presents to the dendrochronology community one such method, Cross-Section Tree-Ring Detection (CS-TRD), which identifies and marks complete annual rings in cross-sections for tree species presenting a clear definition between early and latewood. We compare the CS-TRD performance against the ground truth manual delineation of all rings over the UruDendro dataset. The CS-TRD software identified rings with an average F-score of 89% and RMSE error of 5.27px for the entire database in less than 20 seconds per image. Finally, we propose a robust measure of the ring growth using the \emph{equivalent radius} of a circle having the same area enclosed by the detected tree ring. Overall, this study contributes to the dendrochronologist's toolbox of fast and low-cost methods to automatically detect rings in conifer species, particularly for measuring diameter growth rates and stem transverse area using entire cross-sections.
Forcing Diffuse Distributions out of Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Despite being trained specifically to follow user instructions, today's instructiontuned language models perform poorly when instructed to produce random outputs. For example, when prompted to pick a number uniformly between one and ten Llama-2-13B-chat disproportionately favors the number five, and when tasked with picking a first name at random, Mistral-7B-Instruct chooses Avery 40 times more often than we would expect based on the U.S. population. When these language models are used for real-world tasks where diversity of outputs is crucial, such as language model assisted dataset construction, their inability to produce diffuse distributions over valid choices is a major hurdle. In this work, we propose a fine-tuning method that encourages language models to output distributions that are diffuse over valid outcomes. The methods we introduce generalize across a variety of tasks and distributions and make large language models practical for synthetic dataset generation with little human intervention.
CrossGP: Cross-Day Glucose Prediction Excluding Physiological Information
Venue: arXiv cs.AI 2024
Year: 2024

The increasing number of diabetic patients is a serious issue in society today, which has significant negative impacts on people's health and the country's financial expenditures. Because diabetes may develop into potential serious complications, early glucose prediction for diabetic patients is necessary for timely medical treatment. Existing glucose prediction methods typically utilize patients' private data (e.g. age, gender, ethnicity) and physiological parameters (e.g. blood pressure, heart rate) as reference features for glucose prediction, which inevitably leads to privacy protection concerns. Moreover, these models generally focus on either long-term (monthly-based) or short-term (minute-based) predictions. Long-term prediction methods are generally inaccurate because of the external uncertainties that can greatly affect the glucose values, while short-term ones fail to provide timely medical guidance. Based on the above issues, we propose CrossGP, a novel machine-learning framework for cross-day glucose prediction solely based on the patient's external activities without involving any physiological parameters. Meanwhile, we implement three baseline models for comparison. Extensive experiments on Anderson's dataset strongly demonstrate the superior performance of CrossGP and prove its potential for future real-life applications.
Teaching a Multilingual Large Language Model to Understand Multilingual Speech via Multi-Instructional Training
Venue: arXiv cs.CL 2024
Year: 2024

Recent advancements in language modeling have led to the emergence of Large Language Models (LLMs) capable of various natural language processing tasks. Despite their success in text-based tasks, applying LLMs to the speech domain remains limited and challenging. This paper presents BLOOMZMMS, a novel model that integrates a multilingual LLM with a multilingual speech encoder, aiming to harness the capabilities of LLMs for speech recognition and beyond. Utilizing a multi-instructional training approach, we demonstrate the transferability of linguistic knowledge from the text to the speech modality. Our experiments, conducted on 1900 hours of transcribed data from 139 languages, establish that a multilingual speech representation can be effectively learned and aligned with a multilingual LLM. While this learned representation initially shows limitations in task generalization, we address this issue by generating synthetic targets in a multi-instructional style. Our zero-shot evaluation results confirm the robustness of our approach across multiple tasks, including speech translation and multilingual spoken language understanding, thereby opening new avenues for applying LLMs in the speech domain.
GPU-Based Parallel Computing Methods for Medical Photoacoustic Image Reconstruction
Venue: arXiv cs.DC 2024
Year: 2024

Recent years have witnessed a rapid advancement in GPU technology, establishing it as a formidable high-performance parallel computing technology with superior floating-point computational capabilities compared to traditional CPUs. This paper explores the application of this technology in the field of photoacoustic imaging, an emerging non-destructive testing technique in biomedical engineering characterized by its high contrast, resolution, and penetration depth. We conduct a data parallelism analysis targeting the computationally intensive image reconstruction segment of photoacoustic imaging. By parallelizing the serial code for iterative reconstruction and optimizing memory access, we achieve significant improvements in processing speed. Our experiments compare the imaging speeds of vascular images reconstructed using CPUs and GPUs, with the results visualized using Matlab. The findings demonstrate that, while maintaining data accuracy, GPU parallel computing methods can markedly accelerate photoacoustic image reconstruction. This acceleration has the potential to facilitate the broader adoption of photoacoustic imaging in applications such as hemodynamic monitoring, clinical disease diagnosis, and drug development.
Decreasing Wages in Gig Economy: A Game Theoretic Explanation Using Mathematical Program Networks
Venue: arXiv cs.GT 2024
Year: 2024

Gig economy consists of two market groups connected via an intermediary. Popular examples are rideshares where passengers and drivers are mediated via platforms such as Uber and Lyft. In a duopoly market, the platforms must compete to attract not only the passengers by providing a lower rate but also the drivers by providing better wages. While this should indicate better driver payout, as platforms compete to attract the driver pool, real world statistics does not indicate such. This goes completely against the intuition that the worker side of a gig economy, given their importance, should always earn better. We attempt to answer the low wages of drivers in the gig economy by modeling the ridesharing game between duopoly platforms, drivers, and passengers using Mathematical Program Networks. Our model is parsimonious, expressive, models the same-side and cross-side externalities of the economy, and has interpretations under both single-homing and multi-homing regimes. We derive the conditions for the existence of a profitable duopoly and show that it can only happen if the platforms collude together to pay the bare minimum to the drivers. This not only answers why drivers are paid less but also provides strong managerial insights to any interested policy maker.
Integrated Communication, Navigation, and Remote Sensing in LEO Networks with Vehicular Applications
Venue: arXiv cs.IT 2024
Year: 2024

Traditionally, communication, navigation, and remote sensing (CNR) satellites are separately performed, leading to resource waste, information isolation, and independent optimization for each functionality. Taking future automated driving as an example, it faces great challenges in providing high-reliable and low-latency lane-level positioning, decimeter-level transportation observation, and huge traffic sensing information downloading. To this end, this article proposes an integrated CNR (ICNR) framework based on low Earth orbit (LEO) satellite mega-constellations. After introducing the main working principles of the CNR functionalities to serve as the technological basis, we characterize the potentials of the integration gain in vehicular use cases. Then, we investigate the ICNR framework in different integration levels, which sheds strong light on qualitative performance improvement by sophisticatedly sharing orbit constellation, wireless resource, and data information towards meeting the requirements of vehicular applications. We also instantiate a fundamental numerical case study to demonstrate the integration gain and highlight possible future research directions in managing the ICNR networks.
A Survey on Retrieval-Augmented Text Generation for Large Language Models
Venue: arXiv cs.IR 2024
Year: 2024

Retrieval-Augmented Generation (RAG) merges retrieval methods with deep learning advancements to address the static limitations of large language models (LLMs) by enabling the dynamic integration of up-to-date external information. This methodology, focusing primarily on the text domain, provides a cost-effective solution to the generation of plausible but possibly incorrect responses by LLMs, thereby enhancing the accuracy and reliability of their outputs through the use of real-world data. As RAG grows in complexity and incorporates multiple concepts that can influence its performance, this paper organizes the RAG paradigm into four categories: pre-retrieval, retrieval, post-retrieval, and generation, offering a detailed perspective from the retrieval viewpoint. It outlines RAG's evolution and discusses the field's progression through the analysis of significant studies. Additionally, the paper introduces evaluation methods for RAG, addressing the challenges faced and proposing future research directions. By offering an organized framework and categorization, the study aims to consolidate existing research on RAG, clarify its technological underpinnings, and highlight its potential to broaden the adaptability and applications of LLMs.
Automating Personalized Parsons Problems with Customized Contexts and Concepts
Venue: arXiv cs.CY 2024
Year: 2024

Parsons problems provide useful scaffolding for introductory programming students learning to write code. However, generating large numbers of high-quality Parsons problems that appeal to the diverse range of interests in a typical introductory course is a significant challenge for educators. Large language models (LLMs) may offer a solution, by allowing students to produce on-demand Parsons problems for topics covering the breadth of the introductory programming curriculum, and targeting thematic contexts that align with their personal interests. In this paper, we introduce PuzzleMakerPy, an educational tool that uses an LLM to generate unlimited contextualized drag-and-drop programming exercises in the form of Parsons Problems, which introductory programmers can use as a supplemental learning resource. We evaluated PuzzleMakerPy by deploying it in a large introductory programming course, and found that the ability to personalize the contextual framing used in problem descriptions was highly engaging for students, and being able to customize the programming topics was reported as being useful for their learning.