OpenRaiser/Intern-Atlas huggingface dataset:
Reliability Analysis of Fault Tolerant Memory Systems
Venue: arXiv cs.DC 2023
Year: 2023

This paper delves into a comprehensive analysis of fault-tolerant memory systems, focusing on recovery techniques modeled using Markov chains to address transient errors. The study revolves around the application of scrubbing methods in conjunction with Single Error Correction and Double Error Detection (SEC-DED) codes. It explores three primary models: 1) Exponentially distributed scrubbing, involving periodic checks of memory words within exponentially distributed time intervals; 2) Deterministic scrubbing, featuring regular, periodic word checks; and 3) Mixed scrubbing, which combines both probabilistic and deterministic scrubbing approaches. The research encompasses the estimation of reliability and Mean Time to Failure (MTTF) values for each model. Notably, the findings highlight the superior performance of mixed scrubbing over simpler scrubbing methods in terms of reliability and MTTF.
Cell-free Terahertz Networks: A Spatial-spectral Approach
Venue: arXiv cs.IT 2023
Year: 2023

Cell-free network architecture plays a promising role in the terahertz (THz) networks since it provides better link reliability and uniformly good services for all the users compared to the co-located massive MIMO counterpart, and the spatial-spectral THz link has the advantages of lower initial access latency and fast beam operations. To this end, this work studies cell-free spatial-spectral THz networks with leaky-wave antennas, to exploit the benefits of leveraging both cell-free and spatial-spectral THz technologies. By addressing the coupling effects between propagation angles and frequencies, we propose novel frequency-dependent THz transmit antenna selection schemes to maximize the transmission rate. Numerical results confirm that the proposed antenna selection schemes can achieve much larger transmission rate than the maximal ratio transmission of using all the transmit antennas with equal subchannel bandwidth allocation in higher THz frequencies.
Contextualised Out-of-Distribution Detection using Pattern Identication
Venue: arXiv cs.CV 2023
Year: 2023

In this work, we propose CODE, an extension of existing work from the field of explainable AI that identifies class-specific recurring patterns to build a robust Out-of-Distribution (OoD) detection method for visual classifiers. CODE does not require any classifier retraining and is OoD-agnostic, i.e., tuned directly to the training dataset. Crucially, pattern identification allows us to provide images from the In-Distribution (ID) dataset as reference data to provide additional context to the confidence scores. In addition, we introduce a new benchmark based on perturbations of the ID dataset that provides a known and quantifiable measure of the discrepancy between the ID and OoD datasets serving as a reference value for the comparison between OoD detection methods.
RAEDiff: Denoising Diffusion Probabilistic Models Based Reversible Adversarial Examples Self-Generation and Self-Recovery
Venue: arXiv cs.CR 2023
Year: 2023

Collected and annotated datasets, which are obtained through extensive efforts, are effective for training Deep Neural Network (DNN) models. However, these datasets are susceptible to be misused by unauthorized users, resulting in infringement of Intellectual Property (IP) rights owned by the dataset creators. Reversible Adversarial Exsamples (RAE) can help to solve the issues of IP protection for datasets. RAEs are adversarial perturbed images that can be restored to the original. As a cutting-edge approach, RAE scheme can serve the purposes of preventing unauthorized users from engaging in malicious model training, as well as ensuring the legitimate usage of authorized users. Nevertheless, in the existing work, RAEs still rely on the embedded auxiliary information for restoration, which may compromise their adversarial abilities. In this paper, a novel self-generation and self-recovery method, named as RAEDiff, is introduced for generating RAEs based on a Denoising Diffusion Probabilistic Models (DDPM). It diffuses datasets into a Biased Gaussian Distribution (BGD) and utilizes the prior knowledge of the DDPM for generating and recovering RAEs. The experimental results demonstrate that RAEDiff effectively self-generates adversarial perturbations for DNN models, including Artificial Intelligence Generated Content (AIGC) models, while also exhibiting significant self-recovery capabilities.
Joint Multi-View Collaborative Clustering
Venue: arXiv cs.CV 2023
Year: 2023

Data is increasingly being collected from multiple sources and described by multiple views. These multi-view data provide richer information than traditional single-view data. Fusing the former for specific tasks is an essential component of multi-view clustering. Since the goal of multi-view clustering algorithms is to discover the common latent structure shared by multiple views, the majority of proposed solutions overlook the advantages of incorporating knowledge derived from horizontal collaboration between multi-view data and the final consensus. To fill this gap, we propose the Joint Multi-View Collaborative Clustering (JMVCC) solution, which involves the generation of basic partitions using Non-negative Matrix Factorization (NMF) and the horizontal collaboration principle, followed by the fusion of these local partitions using ensemble clustering. Furthermore, we propose a weighting method to reduce the risk of negative collaboration (i.e., views with low quality) during the generation and fusion of local partitions. The experimental results, which were obtained using a variety of data sets, demonstrate that JMVCC outperforms other multi-view clustering algorithms and is robust to noisy views.
LLVM Static Analysis for Program Characterization and Memory Reuse Profile Estimation
Venue: arXiv cs.SE 2023
Year: 2023

Profiling various application characteristics, including the number of different arithmetic operations performed, memory footprint, etc., dynamically is time- and space-consuming. On the other hand, static analysis methods, although fast, can be less accurate. This paper presents an LLVM-based probabilistic static analysis method that accurately predicts different program characteristics and estimates the reuse distance profile of a program by analyzing the LLVM IR file in constant time, regardless of program input size. We generate the basic-block-level control flow graph of the target application kernel and determine basic-block execution counts by solving the linear balance equation involving the adjacent basic blocks' transition probabilities. Finally, we represent the kernel memory accesses in a bracketed format and employ a recursive algorithm to calculate the reuse distance profile. The results show that our approach can predict application characteristics accurately compared to another LLVM-based dynamic code analysis tool, Byfl.
Diffusion Model Alignment Using Direct Preference Optimization
Venue: arXiv cs.CV 2023
Year: 2023

Large language models (LLMs) are fine-tuned using human comparison data with Reinforcement Learning from Human Feedback (RLHF) methods to make them better aligned with users' preferences. In contrast to LLMs, human preference learning has not been widely explored in text-to-image diffusion models; the best existing approach is to fine-tune a pretrained model using carefully curated high quality images and captions to improve visual appeal and text alignment. We propose Diffusion-DPO, a method to align diffusion models to human preferences by directly optimizing on human comparison data. Diffusion-DPO is adapted from the recently developed Direct Preference Optimization (DPO), a simpler alternative to RLHF which directly optimizes a policy that best satisfies human preferences under a classification objective. We re-formulate DPO to account for a diffusion model notion of likelihood, utilizing the evidence lower bound to derive a differentiable objective. Using the Pick-a-Pic dataset of 851K crowdsourced pairwise preferences, we fine-tune the base model of the state-of-the-art Stable Diffusion XL (SDXL)-1.0 model with Diffusion-DPO. Our fine-tuned base model significantly outperforms both base SDXL-1.0 and the larger SDXL-1.0 model consisting of an additional refinement model in human evaluation, improving visual appeal and prompt alignment. We also develop a variant that uses AI feedback and has comparable performance to training on human preferences, opening the door for scaling of diffusion model alignment methods.
A web-based gamification of upper extremity robotic rehabilitation
Venue: arXiv cs.RO 2023
Year: 2023

In recent years, gamification has become very popular for rehabilitating different cognitive and motor problems. It has been shown that rehabilitation is effective when it starts early enough and it is intensive and repetitive. However, the success of rehabilitation depends also on the motivation and perseverance of patients during treatment. Adding serious games to the rehabilitation procedure will help the patients to overcome the monotonicity of the treatment procedure. On the other hand, if a variety of games can be used with a robotic rehabilitation system, it will help to define tasks with different levels of difficulty with greater variety. In this paper we introduce a procedure for connecting a rehabilitation robot to several web-based games. In other words, an interface is designed that connects the robot to a computer through a USB port. To validate the usefulness of the proposed approach, a researcher designed survey was used to get feedback from several users. The results demonstrate that having several games besides rehabilitation makes the procedure of rehabilitation entertaining.
GAIA: a benchmark for General AI Assistants
Venue: arXiv cs.CL 2023
Year: 2023

We introduce GAIA, a benchmark for General AI Assistants that, if solved, would represent a milestone in AI research. GAIA proposes real-world questions that require a set of fundamental abilities such as reasoning, multi-modality handling, web browsing, and generally tool-use proficiency. GAIA questions are conceptually simple for humans yet challenging for most advanced AIs: we show that human respondents obtain 92\% vs. 15\% for GPT-4 equipped with plugins. This notable performance disparity contrasts with the recent trend of LLMs outperforming humans on tasks requiring professional skills in e.g. law or chemistry. GAIA's philosophy departs from the current trend in AI benchmarks suggesting to target tasks that are ever more difficult for humans. We posit that the advent of Artificial General Intelligence (AGI) hinges on a system's capability to exhibit similar robustness as the average human does on such questions. Using GAIA's methodology, we devise 466 questions and their answer. We release our questions while retaining answers to 300 of them to power a leader-board available at https://huggingface.co/gaia-benchmark.
A Controlled Study on Evaluation of Thermal Stimulation Influence on Affective Measures of Uninformed Individuals
Venue: arXiv cs.HC 2023
Year: 2023

Although the relationship between temperature and emotional states has been investigated in the field of haptics, it remains unknown if, or in what direction, temperature affects emotional states. We approach this question at the intersection of haptics and psychology using a custom-built thermal device and emotional responses based on photos from the International Affective Picture System (IAPS) library. Unlike past research, this study incorporates deception and a control (i.e., neutral temperature) condition. One hundred and twenty naive subjects reported their emotional responses to fifty-six images varying on normative arousal and valence ratings while being exposed to a cool~(30°C), neutral (33°C), or warm (36°C) temperature applied to the upper back. Participants exposed to warm temperatures reported higher arousal ratings in some image categories than participants exposed to neutral or cool temperatures. Valence ratings were decreased in warm conditions compared to neutral conditions. The emotion wheel was used as a complementary method of affective response measurement, and exploratory analysis methods were implemented. Although the valence and arousal showed statistical significance, the emotion wheel results did not demonstrate any significant differences between the temperature conditions.
Favour: FAst Variance Operator for Uncertainty Rating
Venue: arXiv cs.LG 2023
Year: 2023

Bayesian Neural Networks (BNN) have emerged as a crucial approach for interpreting ML predictions. By sampling from the posterior distribution, data scientists may estimate the uncertainty of an inference. Unfortunately many inference samples are often needed, the overhead of which greatly hinder BNN's wide adoption. To mitigate this, previous work proposed propagating the first and second moments of the posterior directly through the network. However, on its own this method is even slower than sampling, so the propagated variance needs to be approximated such as assuming independence between neural nodes. The resulting trade-off between quality and inference time did not match even plain Monte Carlo sampling. Our contribution is a more principled variance propagation framework based on "spiked covariance matrices", which smoothly interpolates between quality and inference time. This is made possible by a new fast algorithm for updating a diagonal-plus-low-rank matrix approximation under various operations. We tested our algorithm against sampling based MC Dropout and Variational Inference on a number of downstream uncertainty themed tasks, such as calibration and out-of-distribution testing. We find that Favour is as fast as performing 2-3 inference samples, while matching the performance of 10-100 samples. In summary, this work enables the use of BNN in the realm of performance critical tasks where they have previously been out of reach.
Perceptual Structure in the Absence of Grounding for LLMs: The Impact of Abstractedness and Subjectivity in Color Language
Venue: arXiv cs.CL 2023
Year: 2023

The need for grounding in language understanding is an active research topic. Previous work has suggested that color perception and color language appear as a suitable test bed to empirically study the problem, given its cognitive significance and showing that there is considerable alignment between a defined color space and the feature space defined by a language model. To further study this issue, we collect a large scale source of colors and their descriptions, containing almost a 1 million examples , and perform an empirical analysis to compare two kinds of alignments: (i) inter-space, by learning a mapping between embedding space and color space, and (ii) intra-space, by means of prompting comparatives between color descriptions. Our results show that while color space alignment holds for monolexemic, highly pragmatic color descriptions, this alignment drops considerably in the presence of examples that exhibit elements of real linguistic usage such as subjectivity and abstractedness, suggesting that grounding may be required in such cases.
Optimal Time of Arrival Estimation for MIMO Backscatter Channels
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we propose a novel time of arrival (TOA) estimator for multiple-input-multiple-output (MIMO) backscatter channels in closed form. The proposed estimator refines the estimation precision from the topological structure of the MIMO backscatter channels, and can considerably enhance the estimation accuracy. Particularly, we show that for the general $M \times N$ bistatic topology, the mean square error (MSE) is $\frac{M+N-1}{MN}σ^2_0$, and for the general $M \times M$ monostatic topology, it is $\frac{2M-1}{M^2}σ^2_0$ for the diagonal subchannels, and $\frac{M-1}{M^2}σ^2_0$ for the off-diagonal subchannels, where $σ^2_0$ is the MSE of the conventional least square estimator. In addition, we derive the Cramer-Rao lower bound (CRLB) for MIMO backscatter TOA estimation which indicates that the proposed estimator is optimal. Simulation results verify that the proposed TOA estimator can considerably improve both estimation and positioning accuracy, especially when the MIMO scale is large.
Robust Outlier Bound Condition to Phase Retrieval with Adversarial Sparse Outliers
Venue: arXiv cs.IT 2023
Year: 2023

We consider the problem of recovering an unknown signal $\pmb{x}_0\in \mathbb{R}^{n}$ from phaseless measurements. In this paper, we study the convex phase retrieval problem via PhaseLift from linear Gaussian measurements perturbed by $\ell_{1}$-bounded noise and sparse outliers that can change an adversarially chosen $s$-fraction of the measurement vector. We show that the Robust-PhaseLift model can successfully reconstruct the ground-truth up to global phase for any $s< s^{*}\approx 0.1185$ with $\mathcal{O}(n)$ measurements, even in the case where the sparse outliers may depend on the measurement and the observation. The recovery guarantees are based on the robust outlier bound condition and the analysis of the product of two Gaussian variables. Moreover, we construct adaptive counterexamples to show that the Robust-PhaseLift model fails when $s> s^{*}$ with high probability.
On the Calibration of Large Language Models and Alignment
Venue: arXiv cs.CL 2023
Year: 2023

As large language models attract increasing attention and find widespread application, concurrent challenges of reliability also arise at the same time. Confidence calibration, an effective analysis method for gauging the reliability of deep models, serves as a crucial tool for assessing and improving their reliability. However, such investigation has been comparatively underexplored. In this work, we conduct a systematic examination of the calibration of aligned language models throughout the entire construction process, including pretraining and alignment training. At each stage, we investigate how different training settings, such as parameter scales and training data, affect model calibration. To thoroughly assess model calibration, we evaluate models on three most concerned aspects: generation, factuality and understanding. Our work sheds light on whether popular LLMs are well-calibrated and how the training process influences model calibration.
Trace-enabled Timing Model Synthesis for ROS2-based Autonomous Applications
Venue: arXiv cs.OS 2023
Year: 2023

Autonomous applications are typically developed over Robot Operating System 2.0 (ROS2) even in time-critical systems like automotive. Recent years have seen increased interest in developing model-based timing analysis and schedule optimization approaches for ROS2-based applications. To complement these approaches, we propose a tracing and measurement framework to obtain timing models of ROS2-based applications. It offers a tracer based on extended Berkeley Packet Filter (eBPF) that probes different functions in ROS2 middleware and reads their arguments or return values to reason about the data flow in applications. It combines event traces from ROS2 and the operating system to generate a directed acyclic graph showing ROS2 callbacks, precedence relations between them, and their timing attributes. While being compatible with existing analyses, we also show how to model (i)~message synchronization, e.g., in sensor fusion, and (ii)~service requests from multiple clients, e.g., in motion planning. Considering that, in real-world scenarios, the application code might be confidential and formal models are unavailable, our framework still enables the application of existing analysis and optimization techniques.
An Empirical Study of Uncertainty Estimation Techniques for Detecting Drift in Data Streams
Venue: arXiv cs.LG 2023
Year: 2023

In safety-critical domains such as autonomous driving and medical diagnosis, the reliability of machine learning models is crucial. One significant challenge to reliability is concept drift, which can cause model deterioration over time. Traditionally, drift detectors rely on true labels, which are often scarce and costly. This study conducts a comprehensive empirical evaluation of using uncertainty values as substitutes for error rates in detecting drifts, aiming to alleviate the reliance on labeled post-deployment data. We examine five uncertainty estimation methods in conjunction with the ADWIN detector across seven real-world datasets. Our results reveal that while the SWAG method exhibits superior calibration, the overall accuracy in detecting drifts is not notably impacted by the choice of uncertainty estimation method, with even the most basic method demonstrating competitive performance. These findings offer valuable insights into the practical applicability of uncertainty-based drift detection in real-world, safety-critical applications.
LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenes
Venue: arXiv cs.CV 2023
Year: 2023

With the widespread usage of VR devices and contents, demands for 3D scene generation techniques become more popular. Existing 3D scene generation models, however, limit the target scene to specific domain, primarily due to their training strategies using 3D scan dataset that is far from the real-world. To address such limitation, we propose LucidDreamer, a domain-free scene generation pipeline by fully leveraging the power of existing large-scale diffusion-based generative model. Our LucidDreamer has two alternate steps: Dreaming and Alignment. First, to generate multi-view consistent images from inputs, we set the point cloud as a geometrical guideline for each image generation. Specifically, we project a portion of point cloud to the desired view and provide the projection as a guidance for inpainting using the generative model. The inpainted images are lifted to 3D space with estimated depth maps, composing a new points. Second, to aggregate the new points into the 3D scene, we propose an aligning algorithm which harmoniously integrates the portions of newly generated 3D scenes. The finally obtained 3D scene serves as initial points for optimizing Gaussian splats. LucidDreamer produces Gaussian splats that are highly-detailed compared to the previous 3D scene generation methods, with no constraint on domain of the target scene. Project page: https://luciddreamer-cvlab.github.io/
From Images to Connections: Can DQN with GNNs learn the Strategic Game of Hex?
Venue: arXiv cs.LG 2023
Year: 2023

The gameplay of strategic board games such as chess, Go and Hex is often characterized by combinatorial, relational structures -- capturing distinct interactions and non-local patterns -- and not just images. Nonetheless, most common self-play reinforcement learning (RL) approaches simply approximate policy and value functions using convolutional neural networks (CNN). A key feature of CNNs is their relational inductive bias towards locality and translational invariance. In contrast, graph neural networks (GNN) can encode more complicated and distinct relational structures. Hence, we investigate the crucial question: Can GNNs, with their ability to encode complex connections, replace CNNs in self-play reinforcement learning? To this end, we do a comparison with Hex -- an abstract yet strategically rich board game -- serving as our experimental platform. Our findings reveal that GNNs excel at dealing with long range dependency situations in game states and are less prone to overfitting, but also showing a reduced proficiency in discerning local patterns. This suggests a potential paradigm shift, signaling the use of game-specific structures to reshape self-play reinforcement learning.