OpenRaiser/Intern-Atlas huggingface dataset:
F3T: A soft tactile unit with 3D force and temperature mathematical decoupling ability for robots
Venue: arXiv cs.RO 2024
Year: 2024

The human skin exhibits remarkable capability to perceive contact forces and environmental temperatures, providing intricate information essential for nuanced manipulation. Despite recent advancements in soft tactile sensors, a significant challenge remains in accurately decoupling signals - specifically, separating force from directional orientation and temperature - resulting in fail to meet the advanced application requirements of robots. This research proposes a multi-layered soft sensor unit (F3T) designed to achieve isolated measurements and mathematical decoupling of normal pressure, omnidirectional tangential forces, and temperature. We developed a circular coaxial magnetic film featuring a floating-mountain multi-layer capacitor, facilitating the physical decoupling of normal and tangential forces in all directions. Additionally, we incorporated an ion gel-based temperature sensing film atop the tactile sensor. This sensor is resilient to external pressure and deformation, enabling it to measure temperature and, crucially, eliminate capacitor errors induced by environmental temperature changes. This innovative design allows for the decoupled measurement of multiple signals, paving the way for advancements in higher-level robot motion control, autonomous decision-making, and task planning.
KiloBot: A Programming Language for Deploying Perception-Guided Industrial Manipulators at Scale
Venue: arXiv cs.RO 2024
Year: 2024

We would like industrial robots to handle unstructured environments with cameras and perception pipelines. In contrast to traditional industrial robots that replay offline-crafted trajectories, online behavior planning is required for these perception-guided industrial applications. Aside from perception and planning algorithms, deploying perception-guided manipulators also requires substantial effort in integration. One approach is writing scripts in a traditional language (such as Python) to construct the planning problem and perform integration with other algorithmic modules & external devices. While scripting in Python is feasible for a handful of robots and applications, deploying perception-guided manipulation at scale (e.g., more than 10000 robot workstations in over 2000 customer sites) becomes intractable. To resolve this challenge, we propose a Domain-Specific Language (DSL) for perception-guided manipulation applications. To scale up the deployment,our DSL provides: 1) an easily accessible interface to construct & solve a sub-class of Task and Motion Planning (TAMP) problems that are important in practical applications; and 2) a mechanism to implement flexible control flow to perform integration and address customized requirements of distinct industrial application. Combined with an intuitive graphical programming frontend, our DSL is mainly used by machine operators without coding experience in traditional programming languages. Within hours of training, operators are capable of orchestrating interesting sophisticated manipulation behaviors with our DSL. Extensive practical deployments demonstrate the efficacy of our method.
Revealing Untapped DSP Optimization Potentials for FPGA-Based Systolic Matrix Engines
Venue: arXiv cs.AR 2024
Year: 2024

Systolic architectures are widely embraced by neural network accelerators for their superior performance in highly parallelized computation. The DSP48E2s serve as dedicated arithmetic blocks in Xilinx Ultrascale series FPGAs and constitute a fundamental component in FPGA-based systolic matrix engines. Harnessing the full potential of DSP48E2s in architectural design can result in significant performance enhancements for systolic architectures on Ultrascale series FPGAs. This paper unveils several previously untapped DSP optimization techniques capable of further enhancing FPGA-based systolic matrix engines. We apply these techniques to two well-known systolic architectures: Google TPUv1 and Xilinx Vitis AI DPU. With the proposed techniques, our design achieves substantial resource and power reduction compared to the open-source TPUv1 FPGA implementation and the Vitis AI DPU implementation in the same parallelism setting. We also demonstrate the applicability of our techniques to neuromorphic hardware for supporting spiking neural network acceleration.
Does Subset Sum Admit Short Proofs?
Venue: arXiv cs.DS 2024
Year: 2024

We investigate the question whether Subset Sum can be solved by a polynomial-time algorithm with access to a certificate of length poly(k) where k is the maximal number of bits in an input number. In other words, can it be solved using only few nondeterministic bits? This question has motivated us to initiate a systematic study of certification complexity of parameterized problems. Apart from Subset Sum, we examine problems related to integer linear programming, scheduling, and group theory. We reveal an equivalence class of problems sharing the same hardness with respect to having a polynomial certificate. These include Subset Sum and Boolean Linear Programming parameterized by the number of constraints. Secondly, we present new techniques for establishing lower bounds in this regime. In particular, we show that Subset Sum in permutation groups is at least as hard for nondeterministic computation as 3Coloring in bounded-pathwidth graphs.
CTMBIDS: Convolutional Tsetlin Machine Based Intrusion Detection System for DDoS attacks in an SDN environment
Venue: arXiv cs.CR 2024
Year: 2024

Software Defined Networks (SDN) face many security challenges today. A great deal of research has been done within the field of Intrusion Detection Systems (IDS) in these networks. Yet, numerous approaches still rely on deep learning algorithms. These algorithms suffer from complexity in implementation, high processing power and high memory consumption. In addition to security issues, firstly, the number of datasets that are based on SDN protocols are very small. Secondly, the ones that are available encompass numerous attacks in the network and do not focus on a single attack. For this reason, to introduce an SDN-based IDS with a focus on Distributed Denial of Service (DDoS) attacks, it is necessary to generate a DDoS-oriented dataset whose features can train a high-quality IDS. In this work, in order to address two important challenges in SDNs, initially, we generate three DDoS attack datasets based on three common and different network topologies. In the second step, using the Convolutional Tsetlin Machine (CTM), we introduce a lightweight IDS for DDoS attack dubbed CTMBIDS. The lightweight nature of the CTMBIDS stems from its low memory consumption and also its interpretability compared to the existing complex deep learning models. The low usage of system resources for the CTMBIDS makes it an ideal choice for an optimal software that consumes the SDN controllers least amount of memory. Also, in order to ascertain the quality of the generated datasets, we compare the CTMBIDS empirical results with the DDoS attacks of the KDDCup99 benchmark dataset as well. Since the main focus of this work is on a lightweight IDS, the results show the CTMBIDS performs much more efficiently than deep learning based approaches. Furthermore, the results also show in most datasets, the proposed method has relatively equal or better accuracy and also consumes much less memory than the existing methods.
100 instances is all you need: predicting the success of a new LLM on unseen data by testing on a few instances
Venue: arXiv cs.CL 2024
Year: 2024

Predicting the performance of LLMs on individual task instances is essential to ensure their reliability in high-stakes applications. To do so, a possibility is to evaluate the considered LLM on a set of task instances and train an assessor to predict its performance based on features of the instances. However, this approach requires evaluating each new LLM on a sufficiently large set of task instances to train an assessor specific to it. In this work, we leverage the evaluation results of previously tested LLMs to reduce the number of evaluations required to predict the performance of a new LLM. In practice, we propose to test the new LLM on a small set of reference instances and train a generic assessor which predicts the performance of the LLM on an instance based on the performance of the former on the reference set and features of the instance of interest. We conduct empirical studies on HELM-Lite and KindsOfReasoning, a collection of existing reasoning datasets that we introduce, where we evaluate all instruction-fine-tuned OpenAI models until the January 2024 version of GPT4. When predicting performance on instances with the same distribution as those used to train the generic assessor, we find this achieves performance comparable to the LLM-specific assessors trained on the full set of instances. Additionally, we find that randomly selecting the reference instances performs as well as some advanced selection methods we tested. For out of distribution, however, no clear winner emerges and the overall performance is worse, suggesting that the inherent predictability of LLMs is low.
LLM-based multi-agent poetry generation in non-cooperative environments
Venue: arXiv cs.CL 2024
Year: 2024

Despite substantial progress of large language models (LLMs) for automatic poetry generation, the generated poetry lacks diversity while the training process differs greatly from human learning. Under the rationale that the learning process of the poetry generation systems should be more human-like and their output more diverse and novel, we introduce a framework based on social learning where we emphasize non-cooperative interactions besides cooperative interactions to encourage diversity. Our experiments are the first attempt at LLM-based multi-agent systems in non-cooperative environments for poetry generation employing both TRAINING-BASED agents (GPT-2) and PROMPTING-BASED agents (GPT-3 and GPT-4). Our evaluation based on 96k generated poems shows that our framework benefits the poetry generation process for TRAINING-BASED agents resulting in 1) a 3.0-3.7 percentage point (pp) increase in diversity and a 5.6-11.3 pp increase in novelty according to distinct and novel n-grams. The generated poetry from TRAINING-BASED agents also exhibits group divergence in terms of lexicons, styles and semantics. PROMPTING-BASED agents in our framework also benefit from non-cooperative environments and a more diverse ensemble of models with non-homogeneous agents has the potential to further enhance diversity, with an increase of 7.0-17.5 pp according to our experiments. However, PROMPTING-BASED agents show a decrease in lexical diversity over time and do not exhibit the group-based divergence intended in the social network. Our paper argues for a paradigm shift in creative tasks such as automatic poetry generation to include social learning processes (via LLM-based agent modeling) similar to human interaction.
Practical Forecasting of Cryptocoins Timeseries using Correlation Patterns
Venue: arXiv cs.CE 2024
Year: 2024

Cryptocoins (i.e., Bitcoin, Ether, Litecoin) are tradable digital assets. Ownerships of cryptocoins are registered on distributed ledgers (i.e., blockchains). Secure encryption techniques guarantee the security of the transactions (transfers of coins among owners), registered into the ledger. Cryptocoins are exchanged for specific trading prices. The extreme volatility of such trading prices across all different sets of crypto-assets remains undisputed. However, the relations between the trading prices across different cryptocoins remains largely unexplored. Major coin exchanges indicate trend correlation to advise for sells or buys. However, price correlations remain largely unexplored. We shed some light on the trend correlations across a large variety of cryptocoins, by investigating their coin/price correlation trends over the past two years. We study the causality between the trends, and exploit the derived correlations to understand the accuracy of state-of-the-art forecasting techniques for time series modeling (e.g., GBMs, LSTM and GRU) of correlated cryptocoins. Our evaluation shows (i) strong correlation patterns between the most traded coins (e.g., Bitcoin and Ether) and other types of cryptocurrencies, and (ii) state-of-the-art time series forecasting algorithms can be used to forecast cryptocoins price trends. We released datasets and code to reproduce our analysis to the research community.
Space-Efficient Algorithm for Integer Programming with Few Constraints
Venue: arXiv cs.DS 2024
Year: 2024

Integer linear programs $\min\{c^T x : A x = b, x \in \mathbb{Z}^n_{\ge 0}\}$, where $A \in \mathbb{Z}^{m \times n}$, $b \in \mathbb{Z}^m$, and $c \in \mathbb{Z}^n$, can be solved in pseudopolynomial time for any fixed number of constraints $m = O(1)$. More precisely, in time $(mΔ)^{O(m)} \text{poly}(I)$, where $Δ$ is the maximum absolute value of an entry in $A$ and $I$ the input size. Known algorithms rely heavily on dynamic programming, which leads to a space complexity of similar order of magnitude as the running time. In this paper, we present a polynomial space algorithm that solves integer linear programs in $(mΔ)^{O(m (\log m + \log\logΔ))} \text{poly}(I)$ time, that is, in almost the same time as previous dynamic programming algorithms.
ArtiFade: Learning to Generate High-quality Subject from Blemished Images
Venue: arXiv cs.CV 2024
Year: 2024

Subject-driven text-to-image generation has witnessed remarkable advancements in its ability to learn and capture characteristics of a subject using only a limited number of images. However, existing methods commonly rely on high-quality images for training and may struggle to generate reasonable images when the input images are blemished by artifacts. This is primarily attributed to the inadequate capability of current techniques in distinguishing subject-related features from disruptive artifacts. In this paper, we introduce ArtiFade to tackle this issue and successfully generate high-quality artifact-free images from blemished datasets. Specifically, ArtiFade exploits fine-tuning of a pre-trained text-to-image model, aiming to remove artifacts. The elimination of artifacts is achieved by utilizing a specialized dataset that encompasses both unblemished images and their corresponding blemished counterparts during fine-tuning. ArtiFade also ensures the preservation of the original generative capabilities inherent within the diffusion model, thereby enhancing the overall performance of subject-driven methods in generating high-quality and artifact-free images. We further devise evaluation benchmarks tailored for this task. Through extensive qualitative and quantitative experiments, we demonstrate the generalizability of ArtiFade in effective artifact removal under both in-distribution and out-of-distribution scenarios.
Rethinking Deep Learning: Propagating Information in Neural Networks without Backpropagation and Statistical Optimization
Venue: arXiv cs.LG 2024
Year: 2024

Developing strong AI signifies the arrival of technological singularity, contributing greatly to advancing human civilization and resolving social issues. Neural networks (NNs) and deep learning, which utilize NNs, are expected to lead to strong AI due to their biological neural system-mimicking structures. However, the statistical weight optimization techniques commonly used, such as error backpropagation and loss functions, may hinder the mimicry of neural systems. This study discusses the information propagation capabilities and potential practical applications of NNs as neural system mimicking structures by solving the handwritten character recognition problem in the Modified National Institute of Standards and Technology (MNIST) database without using statistical weight optimization techniques like error backpropagation. In this study, the NNs architecture comprises fully connected layers using step functions as activation functions, with 0-15 hidden layers, and no weight updates. The accuracy is calculated by comparing the average output vectors of the training data for each label with the output vectors of the test data, based on vector similarity. The results showed that the maximum accuracy achieved is around 80%. This indicates that NNs can propagate information correctly without using statistical weight optimization. Additionally, the accuracy decreased with an increasing number of hidden layers. This is attributed to the decrease in the variance of the output vectors as the number of hidden layers increases, suggesting that the output data becomes smooth. This study's NNs and accuracy calculation methods are simple and have room for various improvements. Moreover, creating a feedforward NNs that repeatedly cycles through 'input -> processing -> output -> environmental response -> input -> ...' could pave the way for practical software applications.
Operational Safety in Human-in-the-loop Human-in-the-plant Autonomous Systems
Venue: arXiv cs.HC 2024
Year: 2024

Control affine assumptions, human inputs are external disturbances, in certified safe controller synthesis approaches are frequently violated in operational deployment under causal human actions. This paper takes a human-in-the-loop human-in-the-plant (HIL-HIP) approach towards ensuring operational safety of safety critical autonomous systems: human and real world controller (RWC) are modeled as a unified system. A three-way interaction is considered: a) through personalized inputs and biological feedback processes between HIP and HIL, b) through sensors and actuators between RWC and HIP, and c) through personalized configuration changes and data feedback between HIL and RWC. We extend control Lyapunov theory by generating barrier function (CLBF) under human action plans, model the HIL as a combination of Markov Chain for spontaneous events and Fuzzy inference system for event responses, the RWC as a black box, and integrate the HIL-HIP model with neural architectures that can learn CLBF certificates. We show that synthesized HIL-HIP controller for automated insulin delivery in Type 1 Diabetes is the only controller to meet safety requirements for human action inputs.
The MLIR Transform Dialect. Your compiler is more powerful than you think
Venue: arXiv cs.PL 2024
Year: 2024

To take full advantage of a specific hardware target, performance engineers need to gain control on compilers in order to leverage their domain knowledge about the program and hardware. Yet, modern compilers are poorly controlled, usually by configuring a sequence of coarse-grained monolithic black-box passes, or by means of predefined compiler annotations/pragmas. These can be effective, but often do not let users precisely optimize their varying compute loads. As a consequence, performance engineers have to resort to implementing custom passes for a specific optimization heuristic, requiring compiler engineering expert knowledge. In this paper, we present a technique that provides fine-grained control of general-purpose compilers by introducing the Transform dialect, a controllable IR-based transformation system implemented in MLIR. The Transform dialect empowers performance engineers to optimize their various compute loads by composing and reusing existing - but currently hidden - compiler features without the need to implement new passes or even rebuilding the compiler. We demonstrate in five case studies that the Transform dialect enables precise, safe composition of compiler transformations and allows for straightforward integration with state-of-the-art search methods.
CALM: Cognitive Assessment using Light-insensitive Model
Venue: arXiv cs.HC 2024
Year: 2024

The demand for cognitive load assessment with low-cost easy-to-use equipment is increasing, with applications ranging from safety-critical industries to entertainment. Though pupillometry is an attractive solution for cognitive load estimation in such applications, its sensitivity to light makes it less robust under varying lighting conditions. Multimodal data acquisition provides a viable alternative, where pupillometry is combined with electrocardiography (ECG) or electroencephalography (EEG). In this work, we study the sensitivity of pupillometry-based cognitive load estimation to light. By collecting heart rate variability (HRV) data during the same experimental sessions, we analyze how the multimodal data reduces this sensitivity and increases robustness to light conditions. In addition to this, we compared the performance in multimodal settings using the HRV data obtained from low-cost fitness-grade equipment to that from clinical-grade equipment by synchronously collecting data from both devices for all task conditions. Our results indicate that multimodal data improves the robustness of cognitive load estimation under changes in light conditions and improves the accuracy by more than 20% points over assessment based on pupillometry alone. In addition to that, the fitness grade device is observed to be a potential alternative to the clinical grade one, even in controlled laboratory settings.
Understanding Fairness in Recommender Systems: A Healthcare Perspective
Venue: arXiv cs.LG 2024
Year: 2024

Fairness in AI-driven decision-making systems has become a critical concern, especially when these systems directly affect human lives. This paper explores the public's comprehension of fairness in healthcare recommendations. We conducted a survey where participants selected from four fairness metrics -- Demographic Parity, Equal Accuracy, Equalized Odds, and Positive Predictive Value -- across different healthcare scenarios to assess their understanding of these concepts. Our findings reveal that fairness is a complex and often misunderstood concept, with a generally low level of public understanding regarding fairness metrics in recommender systems. This study highlights the need for enhanced information and education on algorithmic fairness to support informed decision-making in using these systems. Furthermore, the results suggest that a one-size-fits-all approach to fairness may be insufficient, pointing to the importance of context-sensitive designs in developing equitable AI systems.
Experimentation in Content Moderation using RWKV
Venue: arXiv cs.CL 2024
Year: 2024

This paper investigates the RWKV model's efficacy in content moderation through targeted experimentation. We introduce a novel dataset specifically designed for distillation into smaller models, enhancing content moderation practices. This comprehensive dataset encompasses images, videos, sounds, and text data that present societal challenges. Leveraging advanced Large Language Models (LLMs), we generated an extensive set of responses -- 558,958 for text and 83,625 for images -- to train and refine content moderation systems. Our core experimentation involved fine-tuning the RWKV model, capitalizing on its CPU-efficient architecture to address large-scale content moderation tasks. By highlighting the dataset's potential for knowledge distillation, this study not only demonstrates RWKV's capability in improving the accuracy and efficiency of content moderation systems but also paves the way for developing more compact, resource-efficient models in this domain. Datasets and models can be found in HuggingFace: https://huggingface.co/modrwkv