OpenRaiser/Intern-Atlas huggingface dataset:
biquality-learn: a Python library for Biquality Learning
Venue: arXiv cs.LG 2023
Year: 2023

The democratization of Data Mining has been widely successful thanks in part to powerful and easy-to-use Machine Learning libraries. These libraries have been particularly tailored to tackle Supervised Learning. However, strong supervision signals are scarce in practice, and practitioners must resort to weak supervision. In addition to weaknesses of supervision, dataset shifts are another kind of phenomenon that occurs when deploying machine learning models in the real world. That is why Biquality Learning has been proposed as a machine learning framework to design algorithms capable of handling multiple weaknesses of supervision and dataset shifts without assumptions on their nature and level by relying on the availability of a small trusted dataset composed of cleanly labeled and representative samples. Thus we propose biquality-learn: a Python library for Biquality Learning with an intuitive and consistent API to learn machine learning models from biquality data, with well-proven algorithms, accessible and easy to use for everyone, and enabling researchers to experiment in a reproducible way on biquality data.
Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment
Venue: arXiv cs.CL 2023
Year: 2023

Larger language models (LLMs) have taken the world by storm with their massive multi-tasking capabilities simply by optimizing over a next-word prediction objective. With the emergence of their properties and encoded knowledge, the risk of LLMs producing harmful outputs increases, making them unfit for scalable deployment for the public. In this work, we propose a new safety evaluation benchmark RED-EVAL that carries out red-teaming. We show that even widely deployed models are susceptible to the Chain of Utterances-based (CoU) prompting, jailbreaking closed source LLM-based systems such as GPT-4 and ChatGPT to unethically respond to more than 65% and 73% of harmful queries. We also demonstrate the consistency of the RED-EVAL across 8 open-source LLMs in generating harmful responses in more than 86% of the red-teaming attempts. Next, we propose RED-INSTRUCT--An approach for the safety alignment of LLMs. It constitutes two phases: 1) HARMFULQA data collection: Leveraging CoU prompting, we collect a dataset that consists of 1.9K harmful questions covering a wide range of topics, 9.5K safe and 7.3K harmful conversations from ChatGPT; 2) SAFE-ALIGN: We demonstrate how the conversational dataset can be used for the safety alignment of LLMs by minimizing the negative log-likelihood over helpful responses and penalizing over harmful responses by gradient accent over sample loss. Our model STARLING, a fine-tuned Vicuna-7B, is observed to be more safely aligned when evaluated on RED-EVAL and HHH benchmarks while preserving the utility of the baseline models (TruthfulQA, MMLU, and BBH).
A Lightweight Transformer for Faster and Robust EBSD Data Collection
Venue: arXiv cs.CV 2023
Year: 2023

Three dimensional electron back-scattered diffraction (EBSD) microscopy is a critical tool in many applications in materials science, yet its data quality can fluctuate greatly during the arduous collection process, particularly via serial-sectioning. Fortunately, 3D EBSD data is inherently sequential, opening up the opportunity to use transformers, state-of-the-art deep learning architectures that have made breakthroughs in a plethora of domains, for data processing and recovery. To be more robust to errors and accelerate this 3D EBSD data collection, we introduce a two step method that recovers missing slices in an 3D EBSD volume, using an efficient transformer model and a projection algorithm to process the transformer's outputs. Overcoming the computational and practical hurdles of deep learning with scarce high dimensional data, we train this model using only synthetic 3D EBSD data with self-supervision and obtain superior recovery accuracy on real 3D EBSD data, compared to existing methods.
HumanLiff: Layer-wise 3D Human Generation with Diffusion Model
Venue: arXiv cs.CV 2023
Year: 2023

3D human generation from 2D images has achieved remarkable progress through the synergistic utilization of neural rendering and generative models. Existing 3D human generative models mainly generate a clothed 3D human as an undetectable 3D model in a single pass, while rarely considering the layer-wise nature of a clothed human body, which often consists of the human body and various clothes such as underwear, outerwear, trousers, shoes, etc. In this work, we propose HumanLiff, the first layer-wise 3D human generative model with a unified diffusion process. Specifically, HumanLiff firstly generates minimal-clothed humans, represented by tri-plane features, in a canonical space, and then progressively generates clothes in a layer-wise manner. In this way, the 3D human generation is thus formulated as a sequence of diffusion-based 3D conditional generation. To reconstruct more fine-grained 3D humans with tri-plane representation, we propose a tri-plane shift operation that splits each tri-plane into three sub-planes and shifts these sub-planes to enable feature grid subdivision. To further enhance the controllability of 3D generation with 3D layered conditions, HumanLiff hierarchically fuses tri-plane features and 3D layered conditions to facilitate the 3D diffusion model learning. Extensive experiments on two layer-wise 3D human datasets, SynBody (synthetic) and TightCap (real-world), validate that HumanLiff significantly outperforms state-of-the-art methods in layer-wise 3D human generation. Our code will be available at https://skhu101.github.io/HumanLiff.
FineQuant: Unlocking Efficiency with Fine-Grained Weight-Only Quantization for LLMs
Venue: arXiv cs.LG 2023
Year: 2023

Large Language Models (LLMs) have achieved state-of-the-art performance across various language tasks but pose challenges for practical deployment due to their substantial memory requirements. Furthermore, the latest generative models suffer from high inference costs caused by the memory bandwidth bottleneck in the auto-regressive decoding process. To address these issues, we propose an efficient weight-only quantization method that reduces memory consumption and accelerates inference for LLMs. To ensure minimal quality degradation, we introduce a simple and effective heuristic approach that utilizes only the model weights of a pre-trained model. This approach is applicable to both Mixture-of-Experts (MoE) and dense models without requiring additional fine-tuning. To demonstrate the effectiveness of our proposed method, we first analyze the challenges and issues associated with LLM quantization. Subsequently, we present our heuristic approach, which adaptively finds the granularity of quantization, effectively addressing these problems. Furthermore, we implement highly efficient GPU GEMMs that perform on-the-fly matrix multiplication and dequantization, supporting the multiplication of fp16 or bf16 activations with int8 or int4 weights. We evaluate our approach on large-scale open source models such as OPT-175B and internal MoE models, showcasing minimal accuracy loss while achieving up to 3.65 times higher throughput on the same number of GPUs.
Event-based Dynamic Graph Representation Learning for Patent Application Trend Prediction
Venue: arXiv cs.AI 2023
Year: 2023

Accurate prediction of what types of patents that companies will apply for in the next period of time can figure out their development strategies and help them discover potential partners or competitors in advance. Although important, this problem has been rarely studied in previous research due to the challenges in modelling companies' continuously evolving preferences and capturing the semantic correlations of classification codes. To fill in this gap, we propose an event-based dynamic graph learning framework for patent application trend prediction. In particular, our method is founded on the memorable representations of both companies and patent classification codes. When a new patent is observed, the representations of the related companies and classification codes are updated according to the historical memories and the currently encoded messages. Moreover, a hierarchical message passing mechanism is provided to capture the semantic proximities of patent classification codes by updating their representations along the hierarchical taxonomy. Finally, the patent application trend is predicted by aggregating the representations of the target company and classification codes from static, dynamic, and hierarchical perspectives. Experiments on real-world data demonstrate the effectiveness of our approach under various experimental conditions, and also reveal the abilities of our method in learning semantics of classification codes and tracking technology developing trajectories of companies.
An Efficient High-Dimensional Gene Selection Approach based on Binary Horse Herd Optimization Algorithm for Biological Data Classification
Venue: arXiv cs.LG 2023
Year: 2023

The Horse Herd Optimization Algorithm (HOA) is a new meta-heuristic algorithm based on the behaviors of horses at different ages. The HOA was introduced recently to solve complex and high-dimensional problems. This paper proposes a binary version of the Horse Herd Optimization Algorithm (BHOA) in order to solve discrete problems and select prominent feature subsets. Moreover, this study provides a novel hybrid feature selection framework based on the BHOA and a minimum Redundancy Maximum Relevance (MRMR) filter method. This hybrid feature selection, which is more computationally efficient, produces a beneficial subset of relevant and informative features. Since feature selection is a binary problem, we have applied a new Transfer Function (TF), called X-shape TF, which transforms continuous problems into binary search spaces. Furthermore, the Support Vector Machine (SVM) is utilized to examine the efficiency of the proposed method on ten microarray datasets, namely Lymphoma, Prostate, Brain-1, DLBCL, SRBCT, Leukemia, Ovarian, Colon, Lung, and MLL. In comparison to other state-of-the-art, such as the Gray Wolf (GW), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA), the proposed hybrid method (MRMR-BHOA) demonstrates superior performance in terms of accuracy and minimum selected features. Also, experimental results prove that the X-Shaped BHOA approach outperforms others methods.
Reach For the Spheres: Tangency-Aware Surface Reconstruction of SDFs
Venue: arXiv cs.GR 2023
Year: 2023

Signed distance fields (SDFs) are a widely used implicit surface representation, with broad applications in computer graphics, computer vision, and applied mathematics. To reconstruct an explicit triangle mesh surface corresponding to an SDF, traditional isosurfacing methods, such as Marching Cubes and and its variants, are typically used. However, these methods overlook fundamental properties of SDFs, resulting in reconstructions that exhibit severe oversmoothing and feature loss. To address this shortcoming, we propose a novel method based on a key insight: each SDF sample corresponds to a spherical region that must lie fully inside or outside the surface, depending on its sign, and that must be tangent to the surface at some point. Leveraging this understanding, we formulate an energy that gauges the degree of violation of tangency constraints by a proposed surface. We then employ a gradient flow that minimizes our energy, starting from an initial triangle mesh that encapsulates the surface. This algorithm yields superior reconstructions to previous methods, even with sparsely sampled SDFs. Our approach provides a more nuanced understanding of SDFs and offers significant improvements in surface reconstruction.
Enhancing End-to-End Determinism and Reliability in 6TiSCH networks with disjoint leaf-based MPLS-like tunnels
Venue: arXiv cs.NI 2023
Year: 2023

Industrial multi-hop Internet of Things (IIoT) have strict reliability requirements and they are expected to have deterministic behavior. Reliability is associated with the network's ability to provide the best goodput possible to the destination from the source application, while deterministic behavior implies that the packets must also arrive at the destination before the maximum allowable deadline defined by the application expires. Although a relevant number of proposals have arisen in recent years, none of them achieve both restrictions simultaneously. In this work, we propose a cross-layer approach to solve this problem, by combining three strategies: (i) the use of the preferred parents (PP) and alternative parents (AP) together with the PRE (Packet Replication and Elimination) technique at the routing level; (ii) the use of MPLS tunnels from the leafNode, improving the Data Plane, to control the energy consumption and (iii) the use of the BDPC (Bounded Delay Packet Control) algorithm. The combination of the former strategies show that the behavior of the packet flows improves the end-to-end Packet Delivery Rate of the packets arriving before the deadline by 2.04 times with respect to standard Minimum Scheduling Function reference network while simultaneously increasing the minimum average network lifetime by 1.5 times, with respect to the hop by hop uncontrolled usage of PRE.
Minimizing Carbon Footprint for Timely E-Truck Transportation: Hardness and Approximation Algorithm
Venue: arXiv cs.DS 2023
Year: 2023

Carbon footprint optimization (CFO) is important for sustainable heavy-duty e-truck transportation. We consider the CFO problem for timely transportation of e-trucks, where the truck travels from an origin to a destination across a national highway network subject to a deadline. The goal is to minimize the carbon footprint by orchestrating path planning, speed planning, and intermediary charging planning. We first show that it is NP-hard even just to find a feasible CFO solution. We then develop a $(1+ε_F, 1+ε_β)$ bi-criteria approximation algorithm that achieves a carbon footprint within a ratio of $(1+ε_F)$ to the minimum with no deadline violation and at most a ratio of $(1+ε_β)$ battery capacity violation (for any positive $ε_F$ and $ε_β$). Its time complexity is polynomial in the size of the highway network, $1/ε_F$, and $1/ε_β$. Such algorithmic results are among the best possible unless P=NP. Simulation results based on real-world traces show that our scheme reduces up to 11\% carbon footprint as compared to baseline alternatives considering only energy consumption but not carbon footprint.
Towards a High-Performance Object Detector: Insights from Drone Detection Using ViT and CNN-based Deep Learning Models
Venue: arXiv cs.CV 2023
Year: 2023

Accurate drone detection is strongly desired in drone collision avoidance, drone defense and autonomous Unmanned Aerial Vehicle (UAV) self-landing. With the recent emergence of the Vision Transformer (ViT), this critical task is reassessed in this paper using a UAV dataset composed of 1359 drone photos. We construct various CNN and ViT-based models, demonstrating that for single-drone detection, a basic ViT can achieve performance 4.6 times more robust than our best CNN-based transfer learning models. By implementing the state-of-the-art You Only Look Once (YOLO v7, 200 epochs) and the experimental ViT-based You Only Look At One Sequence (YOLOS, 20 epochs) in multi-drone detection, we attain impressive 98% and 96% mAP values, respectively. We find that ViT outperforms CNN at the same epoch, but also requires more training data, computational power, and sophisticated, performance-oriented designs to fully surpass the capabilities of cutting-edge CNN detectors. We summarize the distinct characteristics of ViT and CNN models to aid future researchers in developing more efficient deep learning models.
On the physical layer security capabilities of reconfigurable intelligent surface empowered wireless systems
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we investigate the physical layer security capabilities of reconfigurable intelligent surface (RIS) empowered wireless systems. In more detail, we consider a general system model, in which the links between the transmitter (TX) and the RIS as well as the links between the RIS and the legitimate receiver are modeled as mixture Gamma (MG) random variables (RVs). Moreover, the link between the TX and eavesdropper is also modeled as a MG RV. Building upon this system model, we derive the probability of zero-secrecy capacity as well as the probability of information leakage. Finally, we extract the average secrecy rate for both cases of TX having full and partial channel state information knowledge.
Never Explore Repeatedly in Multi-Agent Reinforcement Learning
Venue: arXiv cs.LG 2023
Year: 2023

In the realm of multi-agent reinforcement learning, intrinsic motivations have emerged as a pivotal tool for exploration. While the computation of many intrinsic rewards relies on estimating variational posteriors using neural network approximators, a notable challenge has surfaced due to the limited expressive capability of these neural statistics approximators. We pinpoint this challenge as the "revisitation" issue, where agents recurrently explore confined areas of the task space. To combat this, we propose a dynamic reward scaling approach. This method is crafted to stabilize the significant fluctuations in intrinsic rewards in previously explored areas and promote broader exploration, effectively curbing the revisitation phenomenon. Our experimental findings underscore the efficacy of our approach, showcasing enhanced performance in demanding environments like Google Research Football and StarCraft II micromanagement tasks, especially in sparse reward settings.
Joint User Association and Transmission Scheduling in Integrated mmWave Access and Terahertz Backhaul Networks
Venue: arXiv cs.IT 2023
Year: 2023

Terahertz wireless backhaul is expected to meet the high-speed backhaul requirements of future ultra-dense networks using millimeter-wave (mmWave) base stations (BSs). In order to achieve higher network capacity with limited resources and meet the quality of service (QoS) requirements of more users in the integrated mmWave access and terahertz backhaul network, this paper formulates a problem of maximizing the number of users successfully served in both the access and backhaul links. Since the problem is a non-linear integer optimization problem, a minimum rate ratio user association and transmission scheduling algorithm is proposed to obtain a suboptimal solution. The proposed algorithm takes the minimum rate ratio as the user association criterion and schedules first the users with fewer backhaul transmission slots. In addition, the algorithm will update the number of access transmission slots allocated to users and the access scheduling results after the backhaul scheduling phase. Numerical results show that the proposed algorithm outperforms several benchmark algorithms in terms of the number of served users and system throughput, and it can cope with a large number of bursty user requests.
GameEval: Evaluating LLMs on Conversational Games
Venue: arXiv cs.CL 2023
Year: 2023

The rapid advancements in large language models (LLMs) have presented challenges in evaluating those models. Existing evaluation methods are either reference-based or preference based, which inevitably need human intervention or introduce test bias caused by evaluator models. In this paper, we propose GameEval, a novel approach to evaluating LLMs through goal-driven conversational games, overcoming the limitations of previous methods. GameEval treats LLMs as game players and assigns them distinct roles with specific goals achieved by launching conversations of various forms, including discussion, question answering, and voting. We design three unique games with cooperative or adversarial objectives, accompanied by corresponding evaluation metrics, to show how this new paradigm comprehensively evaluates model performance.Through extensive experiments, we show that GameEval can effectively differentiate the capabilities of various LLMs, providing a comprehensive assessment of their integrated abilities to solve complex problems. Our public anonymous code is available at https://github.com/GameEval/GameEval.
Clothoid Curve-based Emergency-Stopping Path Planning with Adaptive Potential Field for Autonomous Vehicles
Venue: arXiv cs.RO 2023
Year: 2023

The Potential Field (PF)-based path planning method is widely adopted for autonomous vehicles (AVs) due to its real-time efficiency and simplicity. PF often creates a rigid road boundary, and while this ensures that the ego vehicle consistently operates within the confines of the road, it also brings a lurking peril in emergency scenarios. If nearby vehicles suddenly switch lanes, the AV has to veer off and brake to evade a collision, leading to the "blind alley" effect. In such a situation, the vehicle can become trapped or confused by the conflicting forces from the obstacle vehicle PF and road boundary PF, often resulting in indecision or erratic behavior, even crashes. To address the above-mentioned challenges, this research introduces an Emergency-Stopping Path Planning (ESPP) that incorporates an adaptive PF (APF) and a clothoid curve for urgent evasion. First, we design an emergency triggering estimation to detect the "blind alley" problem by analyzing the PF distribution. Second, we regionalize the driving scene to search the optimal breach point on the road PF and the final stopping point for the vehicle by considering the possible motion range of the obstacle. Finally, we use the optimized clothoid curve to fit these calculated points under vehicle dynamics constraints to generate a smooth emergency avoidance path. The proposed ESPP-based APF method was evaluated by conducting the co-simulation between MATLAB/Simulink and CarSim Simulator in a freeway scene. The simulation results reveal that the proposed method shows increased performance in emergency collision avoidance and renders the vehicle safer, in which the duration of wheel slip is 61.9% shorter, and the maximum steering angle amplitude is 76.9% lower than other potential field-based methods.
Robust Fraud Detection via Supervised Contrastive Learning
Venue: arXiv cs.CR 2023
Year: 2023

Deep learning models have recently become popular for detecting malicious user activity sessions in computing platforms. In many real-world scenarios, only a few labeled malicious and a large amount of normal sessions are available. These few labeled malicious sessions usually do not cover the entire diversity of all possible malicious sessions. In many scenarios, possible malicious sessions can be highly diverse. As a consequence, learned session representations of deep learning models can become ineffective in achieving a good generalization performance for unseen malicious sessions. To tackle this open-set fraud detection challenge, we propose a robust supervised contrastive learning based framework called ConRo, which specifically operates in the scenario where only a few malicious sessions having limited diversity is available. ConRo applies an effective data augmentation strategy to generate diverse potential malicious sessions. By employing these generated and available training set sessions, ConRo derives separable representations w.r.t open-set fraud detection task by leveraging supervised contrastive learning. We empirically evaluate our ConRo framework and other state-of-the-art baselines on benchmark datasets. Our ConRo framework demonstrates noticeable performance improvement over state-of-the-art baselines.